[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:52:00 dist-test-slave-t3q3 krb5kdc[8818](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:52:00 dist-test-slave-t3q3 krb5kdc[8818](info): set up 2 sockets
Aug 09 08:52:00 dist-test-slave-t3q3 krb5kdc[8818](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 08:52:01.374650 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.396s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:52:01 dist-test-slave-t3q3 krb5kdc[8818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:52:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:52:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
I20260809 08:52:01.502912 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:43273
--webserver_interface=127.8.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:44795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:43273
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:01.591804 8841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:01.592015 8841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:01.592039 8841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:01.593242 8841 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:52:01.593287 8841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:01.593302 8841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:01.593315 8841 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:52:01.593328 8841 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:52:01.595052 8841 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:44795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:43273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:43273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:01.595466 8841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:01.595758 8841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 08:52:01.603351 8841 server_base.cc:1034] running on GCE node
W20260809 08:52:01.603416 8852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:01.603353 8850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:01.604274 8849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:01.604615 8841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:01.605024 8841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:01.606204 8841 hybrid_clock.cc:648] HybridClock initialized: now 1786265521606174 us; error 66 us; skew 500 ppm
Aug 09 08:52:01 dist-test-slave-t3q3 krb5kdc[8818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265521, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:01.612725 8841 init.cc:377] Logged in from keytab as kudu/127.8.154.62@KRBTEST.COM (short username kudu)
I20260809 08:52:01.613595 8841 webserver.cc:460] Webserver started at http://127.8.154.62:33087/ using document root <none> and password file <none>
I20260809 08:52:01.613900 8841 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:01.613977 8841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:01.614187 8841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:01.615636 8841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "a1a043c544c645fcbfe37ec6f7e7923c"
format_stamp: "Formatted at 2026-08-09 08:52:01 on dist-test-slave-t3q3"
server_key: "5dbb1119ed5d33a0035b7477a3bea1f0"
server_key_iv: "589deae41977e10a9f3c93e09d296908"
server_key_version: "encryptionkey@0"
I20260809 08:52:01.616405 8841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "a1a043c544c645fcbfe37ec6f7e7923c"
format_stamp: "Formatted at 2026-08-09 08:52:01 on dist-test-slave-t3q3"
server_key: "5dbb1119ed5d33a0035b7477a3bea1f0"
server_key_iv: "589deae41977e10a9f3c93e09d296908"
server_key_version: "encryptionkey@0"
I20260809 08:52:01.618604 8841 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 08:52:01.619725 8862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:01.619992 8841 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:01.620129 8841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "a1a043c544c645fcbfe37ec6f7e7923c"
format_stamp: "Formatted at 2026-08-09 08:52:01 on dist-test-slave-t3q3"
server_key: "5dbb1119ed5d33a0035b7477a3bea1f0"
server_key_iv: "589deae41977e10a9f3c93e09d296908"
server_key_version: "encryptionkey@0"
I20260809 08:52:01.620270 8841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:01.628000 8841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:01.630400 8841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:01.630605 8841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:01.637398 8841 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:43273
I20260809 08:52:01.637493 8958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:43273 every 8 connection(s)
I20260809 08:52:01.638082 8841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:52:01.639519 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 8841
I20260809 08:52:01.639744 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:52:01.639750 8960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:01.640554 8808 external_mini_cluster.cc:1428] Setting key 77913b33c777198a29715e5d89948bda
I20260809 08:52:01.643895 8960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1a043c544c645fcbfe37ec6f7e7923c: Bootstrap starting.
I20260809 08:52:01.644901 8960 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1a043c544c645fcbfe37ec6f7e7923c: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:01.645288 8960 log.cc:824] T 00000000000000000000000000000000 P a1a043c544c645fcbfe37ec6f7e7923c: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:01.646164 8960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1a043c544c645fcbfe37ec6f7e7923c: No bootstrap required, opened a new log
I20260809 08:52:01.648523 8960 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1a043c544c645fcbfe37ec6f7e7923c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1a043c544c645fcbfe37ec6f7e7923c" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 43273 } }
I20260809 08:52:01.648736 8960 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1a043c544c645fcbfe37ec6f7e7923c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:01.648849 8960 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1a043c544c645fcbfe37ec6f7e7923c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1a043c544c645fcbfe37ec6f7e7923c, State: Initialized, Role: FOLLOWER
I20260809 08:52:01.649068 8960 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1a043c544c645fcbfe37ec6f7e7923c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1a043c544c645fcbfe37ec6f7e7923c" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 43273 } }
I20260809 08:52:01.649217 8960 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1a043c544c645fcbfe37ec6f7e7923c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:52:01.649267 8960 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1a043c544c645fcbfe37ec6f7e7923c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:52:01.649341 8960 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1a043c544c645fcbfe37ec6f7e7923c [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:01.649946 8960 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1a043c544c645fcbfe37ec6f7e7923c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1a043c544c645fcbfe37ec6f7e7923c" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 43273 } }
I20260809 08:52:01.650194 8960 leader_election.cc:302] T 00000000000000000000000000000000 P a1a043c544c645fcbfe37ec6f7e7923c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1a043c544c645fcbfe37ec6f7e7923c; no voters:
I20260809 08:52:01.650424 8960 leader_election.cc:288] T 00000000000000000000000000000000 P a1a043c544c645fcbfe37ec6f7e7923c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:52:01.650482 8966 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1a043c544c645fcbfe37ec6f7e7923c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:52:01.650822 8966 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1a043c544c645fcbfe37ec6f7e7923c [term 1 LEADER]: Becoming Leader. State: Replica: a1a043c544c645fcbfe37ec6f7e7923c, State: Running, Role: LEADER
I20260809 08:52:01.650962 8966 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1a043c544c645fcbfe37ec6f7e7923c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1a043c544c645fcbfe37ec6f7e7923c" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 43273 } }
I20260809 08:52:01.650969 8960 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1a043c544c645fcbfe37ec6f7e7923c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:52:01.651837 8970 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1a043c544c645fcbfe37ec6f7e7923c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1a043c544c645fcbfe37ec6f7e7923c. Latest consensus state: current_term: 1 leader_uuid: "a1a043c544c645fcbfe37ec6f7e7923c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1a043c544c645fcbfe37ec6f7e7923c" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 43273 } } }
I20260809 08:52:01.651942 8970 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1a043c544c645fcbfe37ec6f7e7923c [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:01.652082 8967 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1a043c544c645fcbfe37ec6f7e7923c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1a043c544c645fcbfe37ec6f7e7923c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1a043c544c645fcbfe37ec6f7e7923c" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 43273 } } }
I20260809 08:52:01.652182 8967 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1a043c544c645fcbfe37ec6f7e7923c [sys.catalog]: This master's current role is: LEADER
Aug 09 08:52:01 dist-test-slave-t3q3 krb5kdc[8818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265521, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
W20260809 08:52:01.654572 8992 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a1a043c544c645fcbfe37ec6f7e7923c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:52:01.654640 8992 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:52:01.654762 8973 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:52:01.655184 8973 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:52:01.657577 8973 catalog_manager.cc:1347] Generated new cluster ID: 263708a9ef8f44e89de5ea8283b42dad
I20260809 08:52:01.657640 8973 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:52:01.659533 8962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:01.641846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39968 (local address 127.8.154.62:43273)
0809 08:52:01.642095 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:01.642104 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:01.642329 (+ 225us) server_negotiation.cc:407] Connection header received
0809 08:52:01.644470 (+ 2141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:01.644477 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:01.644524 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:01.644670 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:01.646409 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:01.647664 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:01.649132 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:01.649740 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:01.649924 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:01.654712 (+ 4788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:01.654716 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:01.654717 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:01.654730 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:01.657576 (+ 2846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:01.658044 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:01.658047 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:01.658047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:01.658138 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:01.658401 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:01.658403 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:01.658403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:01.658653 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:01.658818 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:01.659102 (+ 284us) server_negotiation.cc:299] Negotiation successful
0809 08:52:01.659270 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":95,"threads_started":1}
I20260809 08:52:01.686390 8973 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:52:01.687292 8973 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:52:01.695511 8973 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1a043c544c645fcbfe37ec6f7e7923c: Generated new TSK 0
I20260809 08:52:01.695757 8973 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab.
Entry for principal kudu/127.8.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab.
I20260809 08:52:01.734720 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.1:0
--local_ip_for_outbound_sockets=127.8.154.1
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:43273
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:44795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:01.829535 9001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:01.829782 9001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:01.829824 9001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:01.830961 9001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:01.831035 9001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:01.831084 9001 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.1
I20260809 08:52:01.832588 9001 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:44795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:43273
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.1
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:01.832926 9001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:01.833189 9001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:01.838857 9008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:01.838891 9011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:01.838884 9013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:01.839061 9001 server_base.cc:1034] running on GCE node
I20260809 08:52:01.839229 9001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:01.839490 9001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:01.840667 9001 hybrid_clock.cc:648] HybridClock initialized: now 1786265521840651 us; error 41 us; skew 500 ppm
Aug 09 08:52:01 dist-test-slave-t3q3 krb5kdc[8818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265521, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:01.843256 9001 init.cc:377] Logged in from keytab as kudu/127.8.154.1@KRBTEST.COM (short username kudu)
I20260809 08:52:01.843855 9001 webserver.cc:460] Webserver started at http://127.8.154.1:33625/ using document root <none> and password file <none>
I20260809 08:52:01.844152 9001 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:01.844223 9001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:01.844388 9001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:01.845541 9001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-0/data/instance:
uuid: "1414f57826de45f4b494e3e13309deb5"
format_stamp: "Formatted at 2026-08-09 08:52:01 on dist-test-slave-t3q3"
server_key: "977773355af75dfbc067a4228983c9a7"
server_key_iv: "1ec7e95d52b919f2e616e17454a68322"
server_key_version: "encryptionkey@0"
I20260809 08:52:01.845995 9001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance:
uuid: "1414f57826de45f4b494e3e13309deb5"
format_stamp: "Formatted at 2026-08-09 08:52:01 on dist-test-slave-t3q3"
server_key: "977773355af75dfbc067a4228983c9a7"
server_key_iv: "1ec7e95d52b919f2e616e17454a68322"
server_key_version: "encryptionkey@0"
I20260809 08:52:01.847646 9001 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:52:01.848519 9024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:01.848702 9001 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 08:52:01.848814 9001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-0/wal
uuid: "1414f57826de45f4b494e3e13309deb5"
format_stamp: "Formatted at 2026-08-09 08:52:01 on dist-test-slave-t3q3"
server_key: "977773355af75dfbc067a4228983c9a7"
server_key_iv: "1ec7e95d52b919f2e616e17454a68322"
server_key_version: "encryptionkey@0"
I20260809 08:52:01.848889 9001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:01.862854 9001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:01.865041 9001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:01.865212 9001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:01.865583 9001 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:01.865972 9001 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:01.866071 9001 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:01.866114 9001 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:01.866159 9001 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:01.878366 9001 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.1:43857
I20260809 08:52:01.878413 9206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.1:43857 every 8 connection(s)
I20260809 08:52:01.878785 9001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
I20260809 08:52:01.881551 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 9001
I20260809 08:52:01.881659 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance
I20260809 08:52:01.881819 8808 external_mini_cluster.cc:1428] Setting key bd5d591f70dd77d1ea4d8e08a3a9e38d
Aug 09 08:52:01 dist-test-slave-t3q3 krb5kdc[8818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265521, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:01.890213 8962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:01.879869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:53485 (local address 127.8.154.62:43273)
0809 08:52:01.880030 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:01.880032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:01.880227 (+ 195us) server_negotiation.cc:407] Connection header received
0809 08:52:01.880809 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:01.880810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:01.880822 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:01.880883 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:01.882255 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:01.883116 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:01.884296 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:01.884709 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:01.884824 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:01.887239 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:01.887241 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:01.887241 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:01.887249 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:01.888714 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:01.889172 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:01.889174 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:01.889174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:01.889241 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:01.889515 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:01.889516 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:01.889516 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:01.889647 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:01.889719 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:01.890019 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 08:52:01.890116 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
I20260809 08:52:01.890422 9211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:01.879830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:43273 (local address 127.8.154.1:53485)
0809 08:52:01.880082 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:01.880133 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:01.880695 (+ 562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:01.880966 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:01.880970 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:01.881847 (+ 877us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:01.882157 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:01.882165 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:01.883254 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:01.883255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:01.884187 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:01.884191 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:01.884824 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:01.884824 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:01.884937 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:01.885568 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:01.885577 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:01.887125 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:01.888819 (+ 1694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:01.888828 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:01.888830 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:01.889075 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:01.889318 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:01.889319 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:01.889319 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:01.889437 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:01.889724 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:01.889732 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:01.889880 (+ 148us) client_negotiation.cc:769] Sending connection context
0809 08:52:01.889993 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 08:52:01.890167 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":102,"threads_started":1}
I20260809 08:52:01.891088 9209 heartbeater.cc:347] Connected to a master server at 127.8.154.62:43273
I20260809 08:52:01.891196 9209 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:01.891402 9209 heartbeater.cc:511] Master 127.8.154.62:43273 requested a full tablet report, sending...
I20260809 08:52:01.892627 8887 ts_manager.cc:194] Registered new tserver with Master: 1414f57826de45f4b494e3e13309deb5 (127.8.154.1:43857)
WARNING: no policy specified for kudu/127.8.154.2@KRBTEST.COM; defaulting to no policy
I20260809 08:52:01.893572 8887 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.1@KRBTEST.COM'} at 127.8.154.1:53485
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab.
Entry for principal kudu/127.8.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab.
I20260809 08:52:01.914764 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.2:0
--local_ip_for_outbound_sockets=127.8.154.2
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:43273
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:44795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:02.003695 9216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:02.003914 9216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:02.003939 9216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:02.005129 9216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:02.005177 9216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:02.005205 9216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.2
I20260809 08:52:02.006592 9216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:44795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:43273
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.2
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:02.006778 9216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:02.006958 9216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:02.012217 9225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:02.012188 9228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:02.012197 9226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:02.012759 9216 server_base.cc:1034] running on GCE node
I20260809 08:52:02.012960 9216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:02.013264 9216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:02.014427 9216 hybrid_clock.cc:648] HybridClock initialized: now 1786265522014404 us; error 34 us; skew 500 ppm
Aug 09 08:52:02 dist-test-slave-t3q3 krb5kdc[8818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265522, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:02.017122 9216 init.cc:377] Logged in from keytab as kudu/127.8.154.2@KRBTEST.COM (short username kudu)
I20260809 08:52:02.017850 9216 webserver.cc:460] Webserver started at http://127.8.154.2:46195/ using document root <none> and password file <none>
I20260809 08:52:02.018132 9216 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:02.018216 9216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:02.018373 9216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:02.019558 9216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-1/data/instance:
uuid: "bc5b44375f0a47af87555565a21dea1c"
format_stamp: "Formatted at 2026-08-09 08:52:02 on dist-test-slave-t3q3"
server_key: "0a0054bb7374efae5a7c47c9a02dbbe5"
server_key_iv: "010cdd1a7851f17b8172bdf1b20a504c"
server_key_version: "encryptionkey@0"
I20260809 08:52:02.020051 9216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance:
uuid: "bc5b44375f0a47af87555565a21dea1c"
format_stamp: "Formatted at 2026-08-09 08:52:02 on dist-test-slave-t3q3"
server_key: "0a0054bb7374efae5a7c47c9a02dbbe5"
server_key_iv: "010cdd1a7851f17b8172bdf1b20a504c"
server_key_version: "encryptionkey@0"
I20260809 08:52:02.021785 9216 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:52:02.022818 9241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:02.023033 9216 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:02.023135 9216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-1/wal
uuid: "bc5b44375f0a47af87555565a21dea1c"
format_stamp: "Formatted at 2026-08-09 08:52:02 on dist-test-slave-t3q3"
server_key: "0a0054bb7374efae5a7c47c9a02dbbe5"
server_key_iv: "010cdd1a7851f17b8172bdf1b20a504c"
server_key_version: "encryptionkey@0"
I20260809 08:52:02.023209 9216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:02.036815 9216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:02.039017 9216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:02.039144 9216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:02.039395 9216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:02.039775 9216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:02.039809 9216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:02.039834 9216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:02.039846 9216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:02.052816 9216 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.2:39889
I20260809 08:52:02.052861 9433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.2:39889 every 8 connection(s)
I20260809 08:52:02.053282 9216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
Aug 09 08:52:02 dist-test-slave-t3q3 krb5kdc[8818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265522, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:02.062284 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 9216
I20260809 08:52:02.062395 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance
I20260809 08:52:02.062573 8808 external_mini_cluster.cc:1428] Setting key 202a7e91595ec58470566de38a0791cf
I20260809 08:52:02.063099 8962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:02.054134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:56293 (local address 127.8.154.62:43273)
0809 08:52:02.054182 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:02.054184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:02.054552 (+ 368us) server_negotiation.cc:407] Connection header received
0809 08:52:02.055050 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:02.055051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:02.055062 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:02.055118 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:02.056128 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:02.056631 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:02.057664 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:02.057853 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:02.057911 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:02.060242 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:02.060243 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:02.060244 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:02.060250 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:02.061601 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:02.061981 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:02.061981 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:02.061982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:02.062035 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:02.062339 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:02.062340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:02.062340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:02.062495 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:02.062548 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:02.062891 (+ 343us) server_negotiation.cc:299] Negotiation successful
0809 08:52:02.063008 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 08:52:02.063261 9439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:02.054121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:43273 (local address 127.8.154.2:56293)
0809 08:52:02.054377 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:02.054415 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:02.054959 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:02.055177 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:02.055181 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:02.055787 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:02.055999 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:02.056004 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:02.056742 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:02.056743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:02.057568 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:02.057570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:02.057955 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:02.057957 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:02.058079 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:02.058692 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:02.058700 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:02.060155 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:02.061682 (+ 1527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:02.061689 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:02.061690 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:02.061915 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:02.062120 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:02.062121 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:02.062121 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:02.062269 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:02.062546 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:02.062553 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:02.062737 (+ 184us) client_negotiation.cc:769] Sending connection context
0809 08:52:02.062845 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 08:52:02.063040 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":182,"threads_started":1}
I20260809 08:52:02.063701 9435 heartbeater.cc:347] Connected to a master server at 127.8.154.62:43273
I20260809 08:52:02.063823 9435 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:02.064020 9435 heartbeater.cc:511] Master 127.8.154.62:43273 requested a full tablet report, sending...
I20260809 08:52:02.064463 8887 ts_manager.cc:194] Registered new tserver with Master: bc5b44375f0a47af87555565a21dea1c (127.8.154.2:39889)
I20260809 08:52:02.064854 8887 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.2@KRBTEST.COM'} at 127.8.154.2:56293
WARNING: no policy specified for kudu/127.8.154.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab.
Entry for principal kudu/127.8.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab.
I20260809 08:52:02.089226 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.3:0
--local_ip_for_outbound_sockets=127.8.154.3
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:43273
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:44795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:02.183000 9442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:02.183207 9442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:02.183233 9442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:02.184412 9442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:02.184454 9442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:02.184480 9442 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.3
I20260809 08:52:02.185941 9442 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:44795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:43273
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9442
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.3
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:02.186198 9442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:02.186445 9442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:02.191987 9450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:02.192018 9449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:02.192018 9454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:02.192203 9442 server_base.cc:1034] running on GCE node
I20260809 08:52:02.192492 9442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:02.192819 9442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:02.194005 9442 hybrid_clock.cc:648] HybridClock initialized: now 1786265522193985 us; error 31 us; skew 500 ppm
Aug 09 08:52:02 dist-test-slave-t3q3 krb5kdc[8818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265522, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:02.196784 9442 init.cc:377] Logged in from keytab as kudu/127.8.154.3@KRBTEST.COM (short username kudu)
I20260809 08:52:02.197449 9442 webserver.cc:460] Webserver started at http://127.8.154.3:39677/ using document root <none> and password file <none>
I20260809 08:52:02.197698 9442 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:02.197746 9442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:02.197921 9442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:02.199048 9442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-2/data/instance:
uuid: "45761feed78e47d19d3b6dc944b65c7d"
format_stamp: "Formatted at 2026-08-09 08:52:02 on dist-test-slave-t3q3"
server_key: "ad54806de8e53d3818332af93798d86b"
server_key_iv: "23003d2ebfd26382c9c4dbc00adad0aa"
server_key_version: "encryptionkey@0"
I20260809 08:52:02.199520 9442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance:
uuid: "45761feed78e47d19d3b6dc944b65c7d"
format_stamp: "Formatted at 2026-08-09 08:52:02 on dist-test-slave-t3q3"
server_key: "ad54806de8e53d3818332af93798d86b"
server_key_iv: "23003d2ebfd26382c9c4dbc00adad0aa"
server_key_version: "encryptionkey@0"
I20260809 08:52:02.201368 9442 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260809 08:52:02.202214 9463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:02.202418 9442 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:52:02.202514 9442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-2/wal
uuid: "45761feed78e47d19d3b6dc944b65c7d"
format_stamp: "Formatted at 2026-08-09 08:52:02 on dist-test-slave-t3q3"
server_key: "ad54806de8e53d3818332af93798d86b"
server_key_iv: "23003d2ebfd26382c9c4dbc00adad0aa"
server_key_version: "encryptionkey@0"
I20260809 08:52:02.202592 9442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:02.227743 9442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:02.230384 9442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:02.230564 9442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:02.230834 9442 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:02.231176 9442 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:02.231232 9442 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:02.231276 9442 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:02.231304 9442 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:02.242214 9442 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.3:36837
I20260809 08:52:02.242260 9662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.3:36837 every 8 connection(s)
I20260809 08:52:02.242619 9442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
I20260809 08:52:02.244905 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 9442
I20260809 08:52:02.245002 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance
I20260809 08:52:02.245163 8808 external_mini_cluster.cc:1428] Setting key 877eaa47c2cf1712321900d31db2f241
Aug 09 08:52:02 dist-test-slave-t3q3 krb5kdc[8818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265522, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:02.252707 8962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:02.243430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:51951 (local address 127.8.154.62:43273)
0809 08:52:02.243490 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:02.243491 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:02.243780 (+ 289us) server_negotiation.cc:407] Connection header received
0809 08:52:02.244411 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:02.244412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:02.244426 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:02.244479 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:02.245583 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:02.246106 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:02.247230 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:02.247435 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:02.247523 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:02.249673 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:02.249675 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:02.249675 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:02.249682 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:02.251000 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:02.251576 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:02.251577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:02.251577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:02.251647 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:02.251972 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:02.251973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:02.251973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:02.252148 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:02.252216 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:02.252561 (+ 345us) server_negotiation.cc:299] Negotiation successful
0809 08:52:02.252643 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 08:52:02.252822 9667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:02.243383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:43273 (local address 127.8.154.3:51951)
0809 08:52:02.243614 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:02.243681 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:02.244292 (+ 611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:02.244558 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:02.244561 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:02.245260 (+ 699us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:02.245480 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:02.245486 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:02.246215 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:02.246216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:02.247117 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:02.247119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:02.247534 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:02.247536 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:02.247670 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:02.248176 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:02.248185 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:02.249581 (+ 1396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:02.251093 (+ 1512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:02.251100 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:02.251109 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:02.251492 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:02.251696 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:02.251697 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:02.251697 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:02.251905 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:02.252220 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:02.252226 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:02.252414 (+ 188us) client_negotiation.cc:769] Sending connection context
0809 08:52:02.252500 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 08:52:02.252629 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":141,"threads_started":1}
I20260809 08:52:02.253363 9664 heartbeater.cc:347] Connected to a master server at 127.8.154.62:43273
I20260809 08:52:02.253492 9664 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:02.253682 9664 heartbeater.cc:511] Master 127.8.154.62:43273 requested a full tablet report, sending...
I20260809 08:52:02.254092 8887 ts_manager.cc:194] Registered new tserver with Master: 45761feed78e47d19d3b6dc944b65c7d (127.8.154.3:36837)
I20260809 08:52:02.254433 8887 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.3@KRBTEST.COM'} at 127.8.154.3:51951
I20260809 08:52:02.257210 8808 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 08:52:02 dist-test-slave-t3q3 krb5kdc[8818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265522, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 08:52:02 dist-test-slave-t3q3 krb5kdc[8818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265522, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.154.1@KRBTEST.COM
I20260809 08:52:02.283738 9676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:02.275834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44332 (local address 127.8.154.1:43857)
0809 08:52:02.276179 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:02.276184 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:02.276195 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:52:02.276224 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:02.276226 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:02.276240 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:02.276303 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:02.276896 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:02.277730 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:02.278455 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:02.278781 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:02.278849 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:02.281112 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:02.281121 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:02.281122 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:02.281128 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:02.282768 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:02.283070 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:02.283073 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:02.283074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:02.283142 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:02.283306 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:02.283306 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:02.283307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:02.283428 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:02.283534 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:02.283591 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 08:52:02.283652 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":203,"threads_started":1}
W20260809 08:52:02.284298 9065 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44332
Aug 09 08:52:02 dist-test-slave-t3q3 krb5kdc[8818](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265522, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 08:52:02 dist-test-slave-t3q3 krb5kdc[8818](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265522, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.154.1@KRBTEST.COM
I20260809 08:52:02.309957 9676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:02.302588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44342 (local address 127.8.154.1:43857)
0809 08:52:02.302688 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:02.302690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:02.302733 (+ 43us) server_negotiation.cc:407] Connection header received
0809 08:52:02.302805 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:02.302806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:02.302820 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:02.302878 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:02.303447 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:02.303954 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:02.304792 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:02.305055 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:02.305122 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:02.307571 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:02.307572 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:02.307572 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:02.307581 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:02.308941 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:02.309264 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:02.309265 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:02.309266 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:02.309320 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:02.309487 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:02.309488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:02.309488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:02.309582 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:02.309675 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:02.309759 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 08:52:02.309830 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 08:52:02.310567 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 9001
I20260809 08:52:02.326689 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 9216
I20260809 08:52:02.342739 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 9442
I20260809 08:52:02.355055 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 8841
2026-08-09T08:52:02Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2415 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:52:02 dist-test-slave-t3q3 krb5kdc[9685](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:52:02 dist-test-slave-t3q3 krb5kdc[9685](info): set up 2 sockets
Aug 09 08:52:02 dist-test-slave-t3q3 krb5kdc[9685](info): commencing operation
krb5kdc: starting...
W20260809 08:52:03.614674 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.230s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:52:03 dist-test-slave-t3q3 krb5kdc[9685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:52:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:52:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
I20260809 08:52:03.740114 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:45585
--webserver_interface=127.8.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:39293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:45585
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:03.827080 9712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:03.827346 9712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:03.827385 9712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:03.828603 9712 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:52:03.828680 9712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:03.828706 9712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:03.828727 9712 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:52:03.828758 9712 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:52:03.830278 9712 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:39293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:45585
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:45585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:03.830569 9712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:03.830807 9712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:03.835969 9724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:03.835963 9727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:03.835963 9730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:03.836372 9712 server_base.cc:1034] running on GCE node
I20260809 08:52:03.836658 9712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:03.836974 9712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:03.838157 9712 hybrid_clock.cc:648] HybridClock initialized: now 1786265523838142 us; error 49 us; skew 500 ppm
Aug 09 08:52:03 dist-test-slave-t3q3 krb5kdc[9685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265523, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:03.840768 9712 init.cc:377] Logged in from keytab as kudu/127.8.154.62@KRBTEST.COM (short username kudu)
I20260809 08:52:03.841352 9712 webserver.cc:460] Webserver started at http://127.8.154.62:43773/ using document root <none> and password file <none>
I20260809 08:52:03.841598 9712 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:03.841665 9712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:03.841818 9712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:03.842998 9712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "57332645370049a3bdd2d288708af3fe"
format_stamp: "Formatted at 2026-08-09 08:52:03 on dist-test-slave-t3q3"
server_key: "68f0d7b7277f50abb4192ed740d6991e"
server_key_iv: "f8eeac43b6af284b243dfcabc92c0cae"
server_key_version: "encryptionkey@0"
I20260809 08:52:03.843381 9712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "57332645370049a3bdd2d288708af3fe"
format_stamp: "Formatted at 2026-08-09 08:52:03 on dist-test-slave-t3q3"
server_key: "68f0d7b7277f50abb4192ed740d6991e"
server_key_iv: "f8eeac43b6af284b243dfcabc92c0cae"
server_key_version: "encryptionkey@0"
I20260809 08:52:03.844898 9712 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:52:03.845703 9737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:03.845892 9712 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:52:03.845999 9712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "57332645370049a3bdd2d288708af3fe"
format_stamp: "Formatted at 2026-08-09 08:52:03 on dist-test-slave-t3q3"
server_key: "68f0d7b7277f50abb4192ed740d6991e"
server_key_iv: "f8eeac43b6af284b243dfcabc92c0cae"
server_key_version: "encryptionkey@0"
I20260809 08:52:03.846081 9712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:03.855288 9712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:03.857296 9712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:03.857411 9712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:03.861701 9712 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:45585
I20260809 08:52:03.861739 9827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:45585 every 8 connection(s)
I20260809 08:52:03.862126 9712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:52:03.863006 9828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:03.865329 9828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57332645370049a3bdd2d288708af3fe: Bootstrap starting.
I20260809 08:52:03.865621 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 9712
I20260809 08:52:03.865722 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:52:03.865872 8808 external_mini_cluster.cc:1428] Setting key 42dafd9d0d557a819e3304fd6afcb334
I20260809 08:52:03.865978 9828 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57332645370049a3bdd2d288708af3fe: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:03.866350 9828 log.cc:824] T 00000000000000000000000000000000 P 57332645370049a3bdd2d288708af3fe: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:03.867147 9828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57332645370049a3bdd2d288708af3fe: No bootstrap required, opened a new log
I20260809 08:52:03.869513 9828 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57332645370049a3bdd2d288708af3fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57332645370049a3bdd2d288708af3fe" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45585 } }
I20260809 08:52:03.869696 9828 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57332645370049a3bdd2d288708af3fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:03.869750 9828 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57332645370049a3bdd2d288708af3fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57332645370049a3bdd2d288708af3fe, State: Initialized, Role: FOLLOWER
I20260809 08:52:03.869885 9828 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57332645370049a3bdd2d288708af3fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57332645370049a3bdd2d288708af3fe" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45585 } }
I20260809 08:52:03.869966 9828 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57332645370049a3bdd2d288708af3fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:52:03.870015 9828 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57332645370049a3bdd2d288708af3fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:52:03.870065 9828 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57332645370049a3bdd2d288708af3fe [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:03.870648 9828 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57332645370049a3bdd2d288708af3fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57332645370049a3bdd2d288708af3fe" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45585 } }
I20260809 08:52:03.870754 9828 leader_election.cc:302] T 00000000000000000000000000000000 P 57332645370049a3bdd2d288708af3fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57332645370049a3bdd2d288708af3fe; no voters:
I20260809 08:52:03.870916 9828 leader_election.cc:288] T 00000000000000000000000000000000 P 57332645370049a3bdd2d288708af3fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:52:03.870983 9837 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57332645370049a3bdd2d288708af3fe [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:52:03.871156 9828 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57332645370049a3bdd2d288708af3fe [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:52:03.871198 9837 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57332645370049a3bdd2d288708af3fe [term 1 LEADER]: Becoming Leader. State: Replica: 57332645370049a3bdd2d288708af3fe, State: Running, Role: LEADER
I20260809 08:52:03.871399 9837 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57332645370049a3bdd2d288708af3fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57332645370049a3bdd2d288708af3fe" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45585 } }
I20260809 08:52:03.871991 9838 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57332645370049a3bdd2d288708af3fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57332645370049a3bdd2d288708af3fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57332645370049a3bdd2d288708af3fe" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45585 } } }
I20260809 08:52:03.872116 9838 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57332645370049a3bdd2d288708af3fe [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:03.872538 9840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57332645370049a3bdd2d288708af3fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57332645370049a3bdd2d288708af3fe. Latest consensus state: current_term: 1 leader_uuid: "57332645370049a3bdd2d288708af3fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57332645370049a3bdd2d288708af3fe" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45585 } } }
I20260809 08:52:03.872651 9851 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:52:03.872691 9840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57332645370049a3bdd2d288708af3fe [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:03.873190 9851 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 08:52:03 dist-test-slave-t3q3 krb5kdc[9685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:03.874967 9851 catalog_manager.cc:1347] Generated new cluster ID: 2f7d43a206a444aab51d529aed8e36e7
I20260809 08:52:03.875026 9851 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:52:03.878513 9832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:03.866438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51246 (local address 127.8.154.62:45585)
0809 08:52:03.866625 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:03.866634 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:03.866670 (+ 36us) server_negotiation.cc:407] Connection header received
0809 08:52:03.867316 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:03.867325 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:03.867386 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:03.867533 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:03.868158 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:03.869622 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:03.870556 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:03.871195 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:03.871655 (+ 460us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:03.874360 (+ 2705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:03.874364 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:03.874365 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:03.874376 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:03.876296 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:03.876688 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:03.876692 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:03.876692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:03.876790 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:03.877009 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:03.877010 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:03.877011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:03.877302 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:03.877488 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:03.878098 (+ 610us) server_negotiation.cc:299] Negotiation successful
0809 08:52:03.878252 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":96,"threads_started":1}
I20260809 08:52:03.897002 9851 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:52:03.897606 9851 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:52:03.902575 9851 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57332645370049a3bdd2d288708af3fe: Generated new TSK 0
I20260809 08:52:03.902796 9851 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab.
Entry for principal kudu/127.8.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab.
I20260809 08:52:03.939627 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.1:0
--local_ip_for_outbound_sockets=127.8.154.1
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:45585
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:39293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:04.043825 9872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:04.044021 9872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:04.044045 9872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:04.045241 9872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:04.045286 9872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:04.045313 9872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.1
I20260809 08:52:04.046804 9872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:39293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:45585
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9872
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.1
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:04.047053 9872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:04.047304 9872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:04.052428 9889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:04.052429 9883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:04.052536 9884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:04.052939 9872 server_base.cc:1034] running on GCE node
I20260809 08:52:04.053228 9872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:04.053478 9872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:04.054617 9872 hybrid_clock.cc:648] HybridClock initialized: now 1786265524054604 us; error 26 us; skew 500 ppm
Aug 09 08:52:04 dist-test-slave-t3q3 krb5kdc[9685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265524, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:04.057299 9872 init.cc:377] Logged in from keytab as kudu/127.8.154.1@KRBTEST.COM (short username kudu)
I20260809 08:52:04.057824 9872 webserver.cc:460] Webserver started at http://127.8.154.1:43367/ using document root <none> and password file <none>
I20260809 08:52:04.058075 9872 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:04.058126 9872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:04.058308 9872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:04.059386 9872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-0/data/instance:
uuid: "5bf5ff1b74b84191b3943649020ba8f7"
format_stamp: "Formatted at 2026-08-09 08:52:04 on dist-test-slave-t3q3"
server_key: "041527bb0c58897d661dc5ac3cd16e26"
server_key_iv: "28b8930e13fa30d8a0852f532208b3b3"
server_key_version: "encryptionkey@0"
I20260809 08:52:04.059787 9872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance:
uuid: "5bf5ff1b74b84191b3943649020ba8f7"
format_stamp: "Formatted at 2026-08-09 08:52:04 on dist-test-slave-t3q3"
server_key: "041527bb0c58897d661dc5ac3cd16e26"
server_key_iv: "28b8930e13fa30d8a0852f532208b3b3"
server_key_version: "encryptionkey@0"
I20260809 08:52:04.061396 9872 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:52:04.062201 9907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:04.062419 9872 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:04.062513 9872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-0/wal
uuid: "5bf5ff1b74b84191b3943649020ba8f7"
format_stamp: "Formatted at 2026-08-09 08:52:04 on dist-test-slave-t3q3"
server_key: "041527bb0c58897d661dc5ac3cd16e26"
server_key_iv: "28b8930e13fa30d8a0852f532208b3b3"
server_key_version: "encryptionkey@0"
I20260809 08:52:04.062592 9872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:04.082160 9872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:04.084005 9872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:04.084218 9872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:04.084463 9872 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:04.084796 9872 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:04.084856 9872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:04.084892 9872 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:04.084918 9872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:04.091220 9872 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.1:34019
I20260809 08:52:04.091264 10098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.1:34019 every 8 connection(s)
I20260809 08:52:04.091647 9872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
I20260809 08:52:04.096652 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 9872
I20260809 08:52:04.096791 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance
I20260809 08:52:04.096973 8808 external_mini_cluster.cc:1428] Setting key 2e3f0d912672a3574c37ef8616fb440c
Aug 09 08:52:04 dist-test-slave-t3q3 krb5kdc[9685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265524, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:04.101863 9832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:04.092497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:42265 (local address 127.8.154.62:45585)
0809 08:52:04.092569 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:04.092571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:04.092797 (+ 226us) server_negotiation.cc:407] Connection header received
0809 08:52:04.093826 (+ 1029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:04.093827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:04.093839 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:04.093901 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:04.094506 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:04.095025 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:04.096093 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:04.096301 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:04.096359 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:04.098684 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:04.098686 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:04.098686 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:04.098694 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:04.100375 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:04.100815 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:04.100816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:04.100816 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:04.100880 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:04.101155 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:04.101163 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:04.101164 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:04.101289 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:04.101366 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:04.101668 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 08:52:04.101766 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 08:52:04.102066 10102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:04.092481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45585 (local address 127.8.154.1:42265)
0809 08:52:04.092653 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:04.092695 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:04.093718 (+ 1023us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:04.093957 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:04.093960 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:04.094228 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:04.094428 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:04.094431 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:04.095119 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:04.095120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:04.095982 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:04.095984 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:04.096387 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:04.096388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:04.096522 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:04.097139 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:04.097149 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:04.098571 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:04.100478 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:04.100487 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:04.100489 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:04.100734 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:04.100953 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:04.100954 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:04.100954 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:04.101075 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:04.101371 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:04.101380 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:04.101526 (+ 146us) client_negotiation.cc:769] Sending connection context
0809 08:52:04.101634 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 08:52:04.101811 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":98,"threads_started":1}
I20260809 08:52:04.102766 10099 heartbeater.cc:347] Connected to a master server at 127.8.154.62:45585
I20260809 08:52:04.102881 10099 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:04.103066 10099 heartbeater.cc:511] Master 127.8.154.62:45585 requested a full tablet report, sending...
I20260809 08:52:04.103642 9767 ts_manager.cc:194] Registered new tserver with Master: 5bf5ff1b74b84191b3943649020ba8f7 (127.8.154.1:34019)
I20260809 08:52:04.104341 9767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.1@KRBTEST.COM'} at 127.8.154.1:42265
WARNING: no policy specified for kudu/127.8.154.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab.
Entry for principal kudu/127.8.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab.
I20260809 08:52:04.123107 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.2:0
--local_ip_for_outbound_sockets=127.8.154.2
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:45585
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:39293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:04.207829 10109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:04.208142 10109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:04.208185 10109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:04.209344 10109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:04.209383 10109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:04.209410 10109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.2
I20260809 08:52:04.210897 10109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:39293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:45585
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.2
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:04.211107 10109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:04.211301 10109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:04.216408 10125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:04.216427 10121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:04.216454 10118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:04.216735 10109 server_base.cc:1034] running on GCE node
I20260809 08:52:04.216929 10109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:04.217175 10109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:04.218345 10109 hybrid_clock.cc:648] HybridClock initialized: now 1786265524218329 us; error 29 us; skew 500 ppm
Aug 09 08:52:04 dist-test-slave-t3q3 krb5kdc[9685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265524, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:04.221012 10109 init.cc:377] Logged in from keytab as kudu/127.8.154.2@KRBTEST.COM (short username kudu)
I20260809 08:52:04.221469 10109 webserver.cc:460] Webserver started at http://127.8.154.2:36343/ using document root <none> and password file <none>
I20260809 08:52:04.221731 10109 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:04.221804 10109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:04.222010 10109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:04.223129 10109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-1/data/instance:
uuid: "868c286f5d734aec92e00ad7c2182bce"
format_stamp: "Formatted at 2026-08-09 08:52:04 on dist-test-slave-t3q3"
server_key: "47aee1558c28d9b987bb329cb379a28a"
server_key_iv: "efa3c1ceac4a2c61365d4d1875900dcf"
server_key_version: "encryptionkey@0"
I20260809 08:52:04.223517 10109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance:
uuid: "868c286f5d734aec92e00ad7c2182bce"
format_stamp: "Formatted at 2026-08-09 08:52:04 on dist-test-slave-t3q3"
server_key: "47aee1558c28d9b987bb329cb379a28a"
server_key_iv: "efa3c1ceac4a2c61365d4d1875900dcf"
server_key_version: "encryptionkey@0"
I20260809 08:52:04.225165 10109 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:52:04.225929 10134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:04.226106 10109 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:52:04.226209 10109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-1/wal
uuid: "868c286f5d734aec92e00ad7c2182bce"
format_stamp: "Formatted at 2026-08-09 08:52:04 on dist-test-slave-t3q3"
server_key: "47aee1558c28d9b987bb329cb379a28a"
server_key_iv: "efa3c1ceac4a2c61365d4d1875900dcf"
server_key_version: "encryptionkey@0"
I20260809 08:52:04.226291 10109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:04.248623 10109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:04.251195 10109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:04.251368 10109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:04.251636 10109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:04.251961 10109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:04.252018 10109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:04.252128 10109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:04.252159 10109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:04.260236 10109 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.2:37775
I20260809 08:52:04.260283 10336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.2:37775 every 8 connection(s)
I20260809 08:52:04.260641 10109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
Aug 09 08:52:04 dist-test-slave-t3q3 krb5kdc[9685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265524, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:04.268821 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 10109
I20260809 08:52:04.268927 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance
I20260809 08:52:04.269160 8808 external_mini_cluster.cc:1428] Setting key 6d84cb7fa602f393ad9118b6995388a0
I20260809 08:52:04.270586 9832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:04.261476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:50243 (local address 127.8.154.62:45585)
0809 08:52:04.261563 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:04.261565 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:04.261752 (+ 187us) server_negotiation.cc:407] Connection header received
0809 08:52:04.262727 (+ 975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:04.262728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:04.262739 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:04.262792 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:04.263432 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:04.263929 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:04.265053 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:04.265310 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:04.265418 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:04.267523 (+ 2105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:04.267524 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:04.267524 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:04.267531 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:04.269017 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:04.269463 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:04.269464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:04.269465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:04.269536 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:04.269833 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:04.269834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:04.269834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:04.269979 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:04.270081 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:04.270381 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 08:52:04.270467 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 08:52:04.270800 10344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:04.261432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45585 (local address 127.8.154.2:50243)
0809 08:52:04.261626 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:04.261664 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:04.262622 (+ 958us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:04.262855 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:04.262858 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:04.263114 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:04.263358 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:04.263362 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:04.264037 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:04.264038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:04.264952 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:04.264954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:04.265435 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:04.265436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:04.265545 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:04.266001 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:04.266009 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:04.267437 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:04.269097 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:04.269103 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:04.269105 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:04.269380 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:04.269608 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:04.269609 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:04.269610 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:04.269717 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:04.270085 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:04.270092 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:04.270245 (+ 153us) client_negotiation.cc:769] Sending connection context
0809 08:52:04.270359 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 08:52:04.270518 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":113,"threads_started":1}
I20260809 08:52:04.271287 10338 heartbeater.cc:347] Connected to a master server at 127.8.154.62:45585
I20260809 08:52:04.271397 10338 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:04.271620 10338 heartbeater.cc:511] Master 127.8.154.62:45585 requested a full tablet report, sending...
I20260809 08:52:04.272131 9767 ts_manager.cc:194] Registered new tserver with Master: 868c286f5d734aec92e00ad7c2182bce (127.8.154.2:37775)
I20260809 08:52:04.272522 9767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.2@KRBTEST.COM'} at 127.8.154.2:50243
WARNING: no policy specified for kudu/127.8.154.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab.
Entry for principal kudu/127.8.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab.
I20260809 08:52:04.296440 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.3:0
--local_ip_for_outbound_sockets=127.8.154.3
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:45585
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:39293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:04.383010 10348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:04.383229 10348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:04.383255 10348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:04.384610 10348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:04.384652 10348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:04.384680 10348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.3
I20260809 08:52:04.386274 10348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:39293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:45585
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.3
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:04.386519 10348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:04.386755 10348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:04.392331 10362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:04.392426 10348 server_base.cc:1034] running on GCE node
W20260809 08:52:04.392299 10355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:04.392305 10365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:04.392724 10348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:04.393004 10348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:04.394184 10348 hybrid_clock.cc:648] HybridClock initialized: now 1786265524394156 us; error 39 us; skew 500 ppm
Aug 09 08:52:04 dist-test-slave-t3q3 krb5kdc[9685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265524, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:04.396947 10348 init.cc:377] Logged in from keytab as kudu/127.8.154.3@KRBTEST.COM (short username kudu)
I20260809 08:52:04.397480 10348 webserver.cc:460] Webserver started at http://127.8.154.3:39999/ using document root <none> and password file <none>
I20260809 08:52:04.397725 10348 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:04.397799 10348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:04.397960 10348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:04.399102 10348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-2/data/instance:
uuid: "af5ff03497974184a4e58662ddaf30b3"
format_stamp: "Formatted at 2026-08-09 08:52:04 on dist-test-slave-t3q3"
server_key: "3400bfb0a545249c1434cd7ff04c169a"
server_key_iv: "8f0725fcc2784e6a68c9d16c6b4097b0"
server_key_version: "encryptionkey@0"
I20260809 08:52:04.399562 10348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance:
uuid: "af5ff03497974184a4e58662ddaf30b3"
format_stamp: "Formatted at 2026-08-09 08:52:04 on dist-test-slave-t3q3"
server_key: "3400bfb0a545249c1434cd7ff04c169a"
server_key_iv: "8f0725fcc2784e6a68c9d16c6b4097b0"
server_key_version: "encryptionkey@0"
I20260809 08:52:04.401166 10348 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:52:04.402081 10383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:04.402316 10348 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:04.402433 10348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-2/wal
uuid: "af5ff03497974184a4e58662ddaf30b3"
format_stamp: "Formatted at 2026-08-09 08:52:04 on dist-test-slave-t3q3"
server_key: "3400bfb0a545249c1434cd7ff04c169a"
server_key_iv: "8f0725fcc2784e6a68c9d16c6b4097b0"
server_key_version: "encryptionkey@0"
I20260809 08:52:04.402516 10348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:04.426004 10348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:04.428270 10348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:04.428447 10348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:04.428720 10348 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:04.429154 10348 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:04.429214 10348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:04.429267 10348 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:04.429296 10348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:04.436419 10348 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.3:36219
I20260809 08:52:04.436548 10581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.3:36219 every 8 connection(s)
I20260809 08:52:04.436856 10348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
I20260809 08:52:04.442489 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 10348
I20260809 08:52:04.442616 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance
I20260809 08:52:04.442800 8808 external_mini_cluster.cc:1428] Setting key 1e2a959a8f6f0eb63e1ee755da663cb0
Aug 09 08:52:04 dist-test-slave-t3q3 krb5kdc[9685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265524, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:04.446568 9832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:04.437773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:32911 (local address 127.8.154.62:45585)
0809 08:52:04.437846 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:04.437849 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:04.438090 (+ 241us) server_negotiation.cc:407] Connection header received
0809 08:52:04.439027 (+ 937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:04.439028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:04.439038 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:04.439097 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:04.439705 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:04.440234 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:04.441227 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:04.441412 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:04.441464 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:04.444104 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:04.444105 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:04.444106 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:04.444113 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:04.445272 (+ 1159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:04.445701 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:04.445702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:04.445702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:04.445752 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:04.446010 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:04.446011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:04.446011 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:04.446136 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:04.446198 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:04.446432 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 08:52:04.446498 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 08:52:04.446776 10585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:04.437770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45585 (local address 127.8.154.3:32911)
0809 08:52:04.437947 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:04.437991 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:04.438931 (+ 940us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:04.439154 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:04.439158 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:04.439421 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:04.439625 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:04.439629 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:04.440314 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:04.440315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:04.441144 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:04.441146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:04.441487 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:04.441488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:04.441593 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:04.442093 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:04.442121 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:04.443989 (+ 1868us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:04.445350 (+ 1361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:04.445356 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:04.445358 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:04.445625 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:04.445808 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:04.445809 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:04.445809 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:04.445927 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:04.446201 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:04.446206 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:04.446322 (+ 116us) client_negotiation.cc:769] Sending connection context
0809 08:52:04.446405 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 08:52:04.446526 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":98,"threads_started":1}
I20260809 08:52:04.447285 10583 heartbeater.cc:347] Connected to a master server at 127.8.154.62:45585
I20260809 08:52:04.447393 10583 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:04.447576 10583 heartbeater.cc:511] Master 127.8.154.62:45585 requested a full tablet report, sending...
I20260809 08:52:04.447978 9767 ts_manager.cc:194] Registered new tserver with Master: af5ff03497974184a4e58662ddaf30b3 (127.8.154.3:36219)
I20260809 08:52:04.448398 9767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.3@KRBTEST.COM'} at 127.8.154.3:32911
I20260809 08:52:04.454633 8808 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:52:04.462716 9832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:04.456822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51248 (local address 127.8.154.62:45585)
0809 08:52:04.456881 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:04.456883 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:04.456975 (+ 92us) server_negotiation.cc:407] Connection header received
0809 08:52:04.457013 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:04.457014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:04.457022 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:04.457075 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:04.457667 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:04.458177 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:04.458919 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:04.459094 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:04.459156 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:04.460312 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:04.460314 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:04.460314 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:04.460320 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:04.461731 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:04.462061 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:04.462062 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:04.462062 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:04.462114 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:04.462287 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:04.462288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:04.462288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:04.462425 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:04.462506 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:04.462591 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 08:52:04.462653 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 08:52:04.465739 9767 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51248:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:52:04.467643 9767 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:52:04.475631 10604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:04.470892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41492 (local address 127.8.154.1:34019)
0809 08:52:04.471306 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:04.471311 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:04.471323 (+ 12us) server_negotiation.cc:407] Connection header received
0809 08:52:04.471373 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:04.471375 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:04.471391 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:04.471513 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:04.471731 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:04.473033 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:04.474405 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:04.475184 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:04.475268 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:04.475335 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:04.475506 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 08:52:04.475546 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":99,"threads_started":1}
I20260809 08:52:04.475665 10598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:04.470674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:34019 (local address 127.0.0.1:41492)
0809 08:52:04.470854 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:04.470893 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:04.470963 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:04.471533 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:04.471537 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:04.471539 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:04.471643 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:04.471645 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:04.473153 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:04.473154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:04.474309 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:04.474312 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:04.475291 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:04.475292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:04.475385 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:04.475400 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 08:52:04.475481 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 08:52:04.475553 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":87,"thread_start_us":53,"threads_started":1}
I20260809 08:52:04.476271 10608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:04.471263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49358 (local address 127.8.154.2:37775)
0809 08:52:04.471475 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:04.471479 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:04.471487 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:52:04.471663 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:04.471666 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:04.471684 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:04.471761 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:04.471996 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:04.473093 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:04.474558 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:04.475232 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:04.475293 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:04.475340 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:04.475786 (+ 446us) server_negotiation.cc:299] Negotiation successful
0809 08:52:04.476181 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":99,"threads_started":1}
I20260809 08:52:04.476505 10599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:04.470734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:36219 (local address 127.0.0.1:59786)
0809 08:52:04.471107 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:04.471131 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:04.471458 (+ 327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:04.471959 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:04.471960 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:04.471962 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:04.471994 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:04.471995 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:04.473071 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:04.473073 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:04.474464 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:04.474467 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:04.476256 (+ 1789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:04.476257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:04.476321 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:04.476325 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:52:04.476384 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 08:52:04.476440 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":49,"thread_start_us":135,"threads_started":1}
I20260809 08:52:04.476621 9986 tablet_service.cc:1467] Processing CreateTablet for tablet 0a4d0d31af9648cd939b5e6e05f297d5 (DEFAULT_TABLE table=test-table [id=fea265a247a44dd0865331d3ca625a4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:04.476758 10601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:04.470851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59786 (local address 127.8.154.3:36219)
0809 08:52:04.471077 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:04.471081 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:04.471426 (+ 345us) server_negotiation.cc:407] Connection header received
0809 08:52:04.471783 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:04.471787 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:04.471802 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:04.471872 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:04.472051 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:04.472961 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:04.475377 (+ 2416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:04.476170 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:04.476243 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:04.476300 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:04.476624 (+ 324us) server_negotiation.cc:299] Negotiation successful
0809 08:52:04.476682 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":113,"threads_started":1}
I20260809 08:52:04.477001 9986 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a4d0d31af9648cd939b5e6e05f297d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:04.477495 10471 tablet_service.cc:1467] Processing CreateTablet for tablet 0a4d0d31af9648cd939b5e6e05f297d5 (DEFAULT_TABLE table=test-table [id=fea265a247a44dd0865331d3ca625a4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:04.477869 10471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a4d0d31af9648cd939b5e6e05f297d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:04.475657 10607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:04.470986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:37775 (local address 127.0.0.1:49358)
0809 08:52:04.471187 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:04.471197 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:04.471261 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:04.471856 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:04.471857 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:04.471859 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:04.471909 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:04.471911 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:04.473312 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:04.473313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:04.474465 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:04.474467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:04.475307 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:04.475307 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:04.475387 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:04.475400 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 08:52:04.475481 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 08:52:04.475553 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"mutex_wait_us":13,"thread_start_us":111,"threads_started":1}
I20260809 08:52:04.481068 10214 tablet_service.cc:1467] Processing CreateTablet for tablet 0a4d0d31af9648cd939b5e6e05f297d5 (DEFAULT_TABLE table=test-table [id=fea265a247a44dd0865331d3ca625a4c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:04.481385 10214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a4d0d31af9648cd939b5e6e05f297d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:04.481554 10610 tablet_bootstrap.cc:492] T 0a4d0d31af9648cd939b5e6e05f297d5 P af5ff03497974184a4e58662ddaf30b3: Bootstrap starting.
I20260809 08:52:04.482216 10610 tablet_bootstrap.cc:654] T 0a4d0d31af9648cd939b5e6e05f297d5 P af5ff03497974184a4e58662ddaf30b3: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:04.482863 10610 log.cc:824] T 0a4d0d31af9648cd939b5e6e05f297d5 P af5ff03497974184a4e58662ddaf30b3: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:04.483608 10610 tablet_bootstrap.cc:492] T 0a4d0d31af9648cd939b5e6e05f297d5 P af5ff03497974184a4e58662ddaf30b3: No bootstrap required, opened a new log
I20260809 08:52:04.483685 10610 ts_tablet_manager.cc:1397] T 0a4d0d31af9648cd939b5e6e05f297d5 P af5ff03497974184a4e58662ddaf30b3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:52:04.484266 10613 tablet_bootstrap.cc:492] T 0a4d0d31af9648cd939b5e6e05f297d5 P 868c286f5d734aec92e00ad7c2182bce: Bootstrap starting.
I20260809 08:52:04.484287 10609 tablet_bootstrap.cc:492] T 0a4d0d31af9648cd939b5e6e05f297d5 P 5bf5ff1b74b84191b3943649020ba8f7: Bootstrap starting.
I20260809 08:52:04.485106 10613 tablet_bootstrap.cc:654] T 0a4d0d31af9648cd939b5e6e05f297d5 P 868c286f5d734aec92e00ad7c2182bce: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:04.485186 10609 tablet_bootstrap.cc:654] T 0a4d0d31af9648cd939b5e6e05f297d5 P 5bf5ff1b74b84191b3943649020ba8f7: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:04.485625 10613 log.cc:824] T 0a4d0d31af9648cd939b5e6e05f297d5 P 868c286f5d734aec92e00ad7c2182bce: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:04.485858 10609 log.cc:824] T 0a4d0d31af9648cd939b5e6e05f297d5 P 5bf5ff1b74b84191b3943649020ba8f7: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:04.486038 10610 raft_consensus.cc:348] T 0a4d0d31af9648cd939b5e6e05f297d5 P af5ff03497974184a4e58662ddaf30b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf5ff1b74b84191b3943649020ba8f7" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 34019 } } peers { permanent_uuid: "af5ff03497974184a4e58662ddaf30b3" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 36219 } } peers { permanent_uuid: "868c286f5d734aec92e00ad7c2182bce" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37775 } }
I20260809 08:52:04.486271 10610 raft_consensus.cc:374] T 0a4d0d31af9648cd939b5e6e05f297d5 P af5ff03497974184a4e58662ddaf30b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:04.486325 10610 raft_consensus.cc:729] T 0a4d0d31af9648cd939b5e6e05f297d5 P af5ff03497974184a4e58662ddaf30b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af5ff03497974184a4e58662ddaf30b3, State: Initialized, Role: FOLLOWER
I20260809 08:52:04.486464 10610 consensus_queue.cc:260] T 0a4d0d31af9648cd939b5e6e05f297d5 P af5ff03497974184a4e58662ddaf30b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf5ff1b74b84191b3943649020ba8f7" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 34019 } } peers { permanent_uuid: "af5ff03497974184a4e58662ddaf30b3" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 36219 } } peers { permanent_uuid: "868c286f5d734aec92e00ad7c2182bce" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37775 } }
I20260809 08:52:04.486536 10613 tablet_bootstrap.cc:492] T 0a4d0d31af9648cd939b5e6e05f297d5 P 868c286f5d734aec92e00ad7c2182bce: No bootstrap required, opened a new log
I20260809 08:52:04.486598 10613 ts_tablet_manager.cc:1397] T 0a4d0d31af9648cd939b5e6e05f297d5 P 868c286f5d734aec92e00ad7c2182bce: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:52:04.486665 10609 tablet_bootstrap.cc:492] T 0a4d0d31af9648cd939b5e6e05f297d5 P 5bf5ff1b74b84191b3943649020ba8f7: No bootstrap required, opened a new log
I20260809 08:52:04.486739 10609 ts_tablet_manager.cc:1397] T 0a4d0d31af9648cd939b5e6e05f297d5 P 5bf5ff1b74b84191b3943649020ba8f7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 08:52:04.486763 10610 ts_tablet_manager.cc:1428] T 0a4d0d31af9648cd939b5e6e05f297d5 P af5ff03497974184a4e58662ddaf30b3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:52:04.486925 10583 heartbeater.cc:503] Master 127.8.154.62:45585 was elected leader, sending a full tablet report...
I20260809 08:52:04.488816 10613 raft_consensus.cc:348] T 0a4d0d31af9648cd939b5e6e05f297d5 P 868c286f5d734aec92e00ad7c2182bce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf5ff1b74b84191b3943649020ba8f7" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 34019 } } peers { permanent_uuid: "af5ff03497974184a4e58662ddaf30b3" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 36219 } } peers { permanent_uuid: "868c286f5d734aec92e00ad7c2182bce" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37775 } }
I20260809 08:52:04.488952 10613 raft_consensus.cc:374] T 0a4d0d31af9648cd939b5e6e05f297d5 P 868c286f5d734aec92e00ad7c2182bce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:04.488837 10609 raft_consensus.cc:348] T 0a4d0d31af9648cd939b5e6e05f297d5 P 5bf5ff1b74b84191b3943649020ba8f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf5ff1b74b84191b3943649020ba8f7" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 34019 } } peers { permanent_uuid: "af5ff03497974184a4e58662ddaf30b3" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 36219 } } peers { permanent_uuid: "868c286f5d734aec92e00ad7c2182bce" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37775 } }
I20260809 08:52:04.488974 10613 raft_consensus.cc:729] T 0a4d0d31af9648cd939b5e6e05f297d5 P 868c286f5d734aec92e00ad7c2182bce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 868c286f5d734aec92e00ad7c2182bce, State: Initialized, Role: FOLLOWER
I20260809 08:52:04.488996 10609 raft_consensus.cc:374] T 0a4d0d31af9648cd939b5e6e05f297d5 P 5bf5ff1b74b84191b3943649020ba8f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:04.489045 10609 raft_consensus.cc:729] T 0a4d0d31af9648cd939b5e6e05f297d5 P 5bf5ff1b74b84191b3943649020ba8f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bf5ff1b74b84191b3943649020ba8f7, State: Initialized, Role: FOLLOWER
I20260809 08:52:04.489091 10613 consensus_queue.cc:260] T 0a4d0d31af9648cd939b5e6e05f297d5 P 868c286f5d734aec92e00ad7c2182bce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf5ff1b74b84191b3943649020ba8f7" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 34019 } } peers { permanent_uuid: "af5ff03497974184a4e58662ddaf30b3" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 36219 } } peers { permanent_uuid: "868c286f5d734aec92e00ad7c2182bce" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37775 } }
I20260809 08:52:04.489166 10609 consensus_queue.cc:260] T 0a4d0d31af9648cd939b5e6e05f297d5 P 5bf5ff1b74b84191b3943649020ba8f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf5ff1b74b84191b3943649020ba8f7" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 34019 } } peers { permanent_uuid: "af5ff03497974184a4e58662ddaf30b3" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 36219 } } peers { permanent_uuid: "868c286f5d734aec92e00ad7c2182bce" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37775 } }
I20260809 08:52:04.489394 10613 ts_tablet_manager.cc:1428] T 0a4d0d31af9648cd939b5e6e05f297d5 P 868c286f5d734aec92e00ad7c2182bce: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:52:04.489405 10338 heartbeater.cc:503] Master 127.8.154.62:45585 was elected leader, sending a full tablet report...
I20260809 08:52:04.489490 10609 ts_tablet_manager.cc:1428] T 0a4d0d31af9648cd939b5e6e05f297d5 P 5bf5ff1b74b84191b3943649020ba8f7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:52:04.489651 10099 heartbeater.cc:503] Master 127.8.154.62:45585 was elected leader, sending a full tablet report...
W20260809 08:52:04.511301 10339 tablet.cc:2369] T 0a4d0d31af9648cd939b5e6e05f297d5 P 868c286f5d734aec92e00ad7c2182bce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:52:04.563040 10618 raft_consensus.cc:482] T 0a4d0d31af9648cd939b5e6e05f297d5 P af5ff03497974184a4e58662ddaf30b3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:52:04.563187 10618 raft_consensus.cc:504] T 0a4d0d31af9648cd939b5e6e05f297d5 P af5ff03497974184a4e58662ddaf30b3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf5ff1b74b84191b3943649020ba8f7" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 34019 } } peers { permanent_uuid: "af5ff03497974184a4e58662ddaf30b3" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 36219 } } peers { permanent_uuid: "868c286f5d734aec92e00ad7c2182bce" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37775 } }
I20260809 08:52:04.563858 10618 leader_election.cc:288] T 0a4d0d31af9648cd939b5e6e05f297d5 P af5ff03497974184a4e58662ddaf30b3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5bf5ff1b74b84191b3943649020ba8f7 (127.8.154.1:34019), 868c286f5d734aec92e00ad7c2182bce (127.8.154.2:37775)
I20260809 08:52:04.567559 10621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:04.564200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:34019 (local address 127.8.154.3:34335)
0809 08:52:04.564394 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:04.564406 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:04.564476 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:04.564634 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:04.564635 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:04.564637 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:04.564698 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:04.564699 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:04.565374 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:04.565375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:04.566730 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:04.566734 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:04.567275 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:04.567277 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:04.567359 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:04.567365 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 08:52:04.567428 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 08:52:04.567463 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":52,"thread_start_us":109,"threads_started":1}
I20260809 08:52:04.567577 10604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:04.564264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:34335 (local address 127.8.154.1:34019)
0809 08:52:04.564306 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:04.564308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:04.564483 (+ 175us) server_negotiation.cc:407] Connection header received
0809 08:52:04.564552 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:04.564553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:04.564566 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:04.564622 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:04.564775 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:04.565250 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:04.566826 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:04.567186 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:04.567254 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:04.567263 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:04.567450 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 08:52:04.567494 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 08:52:04.567672 10585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:04.564127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:37775 (local address 127.8.154.3:40135)
0809 08:52:04.564191 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:04.564203 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:04.564253 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:04.564495 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:04.564496 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:04.564498 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:04.564593 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:04.564595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:04.565374 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:04.565375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:04.566733 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:04.566734 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:04.567375 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:04.567376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:04.567486 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:04.567490 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:52:04.567552 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 08:52:04.567593 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":92}
I20260809 08:52:04.567705 10608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:04.564175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:40135 (local address 127.8.154.2:37775)
0809 08:52:04.564228 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:04.564230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:04.564244 (+ 14us) server_negotiation.cc:407] Connection header received
0809 08:52:04.564350 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:04.564352 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:04.564373 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:04.564439 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:04.564678 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:04.565267 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:04.566864 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:04.567280 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:04.567352 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:04.567362 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:04.567579 (+ 217us) server_negotiation.cc:299] Negotiation successful
0809 08:52:04.567626 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 08:52:04.567999 10241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a4d0d31af9648cd939b5e6e05f297d5" candidate_uuid: "af5ff03497974184a4e58662ddaf30b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "868c286f5d734aec92e00ad7c2182bce" is_pre_election: true
I20260809 08:52:04.568259 10241 raft_consensus.cc:2454] T 0a4d0d31af9648cd939b5e6e05f297d5 P 868c286f5d734aec92e00ad7c2182bce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af5ff03497974184a4e58662ddaf30b3 in term 0.
I20260809 08:52:04.568549 10386 leader_election.cc:302] T 0a4d0d31af9648cd939b5e6e05f297d5 P af5ff03497974184a4e58662ddaf30b3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 868c286f5d734aec92e00ad7c2182bce, af5ff03497974184a4e58662ddaf30b3; no voters:
I20260809 08:52:04.568673 10618 raft_consensus.cc:2781] T 0a4d0d31af9648cd939b5e6e05f297d5 P af5ff03497974184a4e58662ddaf30b3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:52:04.568732 10618 raft_consensus.cc:482] T 0a4d0d31af9648cd939b5e6e05f297d5 P af5ff03497974184a4e58662ddaf30b3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:52:04.568759 10618 raft_consensus.cc:3037] T 0a4d0d31af9648cd939b5e6e05f297d5 P af5ff03497974184a4e58662ddaf30b3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:04.569490 10618 raft_consensus.cc:504] T 0a4d0d31af9648cd939b5e6e05f297d5 P af5ff03497974184a4e58662ddaf30b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf5ff1b74b84191b3943649020ba8f7" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 34019 } } peers { permanent_uuid: "af5ff03497974184a4e58662ddaf30b3" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 36219 } } peers { permanent_uuid: "868c286f5d734aec92e00ad7c2182bce" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37775 } }
I20260809 08:52:04.569641 10618 leader_election.cc:288] T 0a4d0d31af9648cd939b5e6e05f297d5 P af5ff03497974184a4e58662ddaf30b3 [CANDIDATE]: Term 1 election: Requested vote from peers 5bf5ff1b74b84191b3943649020ba8f7 (127.8.154.1:34019), 868c286f5d734aec92e00ad7c2182bce (127.8.154.2:37775)
I20260809 08:52:04.569768 10241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a4d0d31af9648cd939b5e6e05f297d5" candidate_uuid: "af5ff03497974184a4e58662ddaf30b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "868c286f5d734aec92e00ad7c2182bce"
I20260809 08:52:04.569829 10241 raft_consensus.cc:3037] T 0a4d0d31af9648cd939b5e6e05f297d5 P 868c286f5d734aec92e00ad7c2182bce [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:04.569796 10022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a4d0d31af9648cd939b5e6e05f297d5" candidate_uuid: "af5ff03497974184a4e58662ddaf30b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bf5ff1b74b84191b3943649020ba8f7"
I20260809 08:52:04.569903 10022 raft_consensus.cc:3037] T 0a4d0d31af9648cd939b5e6e05f297d5 P 5bf5ff1b74b84191b3943649020ba8f7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:04.567953 10024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a4d0d31af9648cd939b5e6e05f297d5" candidate_uuid: "af5ff03497974184a4e58662ddaf30b3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5bf5ff1b74b84191b3943649020ba8f7" is_pre_election: true
I20260809 08:52:04.570582 10022 raft_consensus.cc:2454] T 0a4d0d31af9648cd939b5e6e05f297d5 P 5bf5ff1b74b84191b3943649020ba8f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af5ff03497974184a4e58662ddaf30b3 in term 1.
I20260809 08:52:04.570779 10241 raft_consensus.cc:2454] T 0a4d0d31af9648cd939b5e6e05f297d5 P 868c286f5d734aec92e00ad7c2182bce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af5ff03497974184a4e58662ddaf30b3 in term 1.
I20260809 08:52:04.570924 10387 leader_election.cc:302] T 0a4d0d31af9648cd939b5e6e05f297d5 P af5ff03497974184a4e58662ddaf30b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5bf5ff1b74b84191b3943649020ba8f7, af5ff03497974184a4e58662ddaf30b3; no voters:
I20260809 08:52:04.571071 10618 raft_consensus.cc:2781] T 0a4d0d31af9648cd939b5e6e05f297d5 P af5ff03497974184a4e58662ddaf30b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:52:04.571218 10618 raft_consensus.cc:686] T 0a4d0d31af9648cd939b5e6e05f297d5 P af5ff03497974184a4e58662ddaf30b3 [term 1 LEADER]: Becoming Leader. State: Replica: af5ff03497974184a4e58662ddaf30b3, State: Running, Role: LEADER
I20260809 08:52:04.571291 10618 consensus_queue.cc:237] T 0a4d0d31af9648cd939b5e6e05f297d5 P af5ff03497974184a4e58662ddaf30b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf5ff1b74b84191b3943649020ba8f7" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 34019 } } peers { permanent_uuid: "af5ff03497974184a4e58662ddaf30b3" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 36219 } } peers { permanent_uuid: "868c286f5d734aec92e00ad7c2182bce" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37775 } }
I20260809 08:52:04.572307 9767 catalog_manager.cc:5495] T 0a4d0d31af9648cd939b5e6e05f297d5 P af5ff03497974184a4e58662ddaf30b3 reported cstate change: term changed from 0 to 1, leader changed from <none> to af5ff03497974184a4e58662ddaf30b3 (127.8.154.3). New cstate: current_term: 1 leader_uuid: "af5ff03497974184a4e58662ddaf30b3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf5ff1b74b84191b3943649020ba8f7" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 34019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af5ff03497974184a4e58662ddaf30b3" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 36219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "868c286f5d734aec92e00ad7c2182bce" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37775 } health_report { overall_health: UNKNOWN } } }
W20260809 08:52:04.592443 10101 tablet.cc:2369] T 0a4d0d31af9648cd939b5e6e05f297d5 P 5bf5ff1b74b84191b3943649020ba8f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 08:52:04 dist-test-slave-t3q3 krb5kdc[9685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.154.1@KRBTEST.COM
I20260809 08:52:04.597038 10604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:04.588977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41508 (local address 127.8.154.1:34019)
0809 08:52:04.589075 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:04.589077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:04.589188 (+ 111us) server_negotiation.cc:407] Connection header received
0809 08:52:04.589282 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:04.589284 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:04.589295 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:04.589347 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:04.589934 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:04.590428 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:04.591171 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:04.591377 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:04.591463 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:04.593954 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:04.593967 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:04.593967 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:04.593975 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:04.595731 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:04.596162 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:04.596166 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:04.596167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:04.596232 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:04.596446 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:04.596447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:04.596448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:04.596600 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:04.596774 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:04.596839 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 08:52:04.596928 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
Aug 09 08:52:04 dist-test-slave-t3q3 krb5kdc[9685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265524, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 08:52:04 dist-test-slave-t3q3 krb5kdc[9685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265524, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.154.1@KRBTEST.COM
I20260809 08:52:04.622187 10604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:04.614841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41510 (local address 127.8.154.1:34019)
0809 08:52:04.614925 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:04.614928 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:04.614994 (+ 66us) server_negotiation.cc:407] Connection header received
0809 08:52:04.615111 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:04.615112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:04.615123 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:04.615190 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:04.615786 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:04.616340 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:04.617138 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:04.617341 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:04.617422 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:04.619740 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:04.619741 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:04.619742 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:04.619749 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:04.621089 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:04.621409 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:04.621410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:04.621410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:04.621460 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:04.621636 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:04.621637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:04.621637 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:04.621740 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:04.621813 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:04.622029 (+ 216us) server_negotiation.cc:299] Negotiation successful
0809 08:52:04.622097 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 08:52:04 dist-test-slave-t3q3 krb5kdc[9685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265524, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 08:52:04 dist-test-slave-t3q3 krb5kdc[9685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265524, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.154.1@KRBTEST.COM
I20260809 08:52:04.648273 10604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:04.640137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41518 (local address 127.8.154.1:34019)
0809 08:52:04.640202 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:04.640205 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:04.640218 (+ 13us) server_negotiation.cc:407] Connection header received
0809 08:52:04.640298 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:04.640299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:04.640314 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:04.640381 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:04.640998 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:04.641496 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:04.642280 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:04.642516 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:04.642600 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:04.644996 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:04.644998 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:04.644998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:04.645006 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:04.646931 (+ 1925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:04.647378 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:04.647380 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:04.647380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:04.647440 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:04.647710 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:04.647711 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:04.647712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:04.647827 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:04.647924 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:04.648085 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 08:52:04.648172 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260809 08:52:04.649019 9961 tablet_service.cc:2908] Rejecting scan request for tablet 0a4d0d31af9648cd939b5e6e05f297d5: Uninitialized: clean time has not yet been initialized
I20260809 08:52:04.650173 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 9872
W20260809 08:52:04.687539 10584 tablet.cc:2369] T 0a4d0d31af9648cd939b5e6e05f297d5 P af5ff03497974184a4e58662ddaf30b3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:52:04.691031 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 10109
I20260809 08:52:04.700534 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 10348
I20260809 08:52:04.714471 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 9712
2026-08-09T08:52:04Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2358 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:52:04 dist-test-slave-t3q3 krb5kdc[10647](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:52:04 dist-test-slave-t3q3 krb5kdc[10647](info): set up 2 sockets
Aug 09 08:52:04 dist-test-slave-t3q3 krb5kdc[10647](info): commencing operation
krb5kdc: starting...
W20260809 08:52:05.984941 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:52:06 dist-test-slave-t3q3 krb5kdc[10647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265526, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:52:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:52:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
I20260809 08:52:06.124936 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:34255
--webserver_interface=127.8.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:34255
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:06.208709 10668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:06.208984 10668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:06.209025 10668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:06.210136 10668 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:52:06.210173 10668 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 08:52:06.210186 10668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:06.210199 10668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:06.210212 10668 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:52:06.210224 10668 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:52:06.211624 10668 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:34255
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:34255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:06.211823 10668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:06.211993 10668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:06.216933 10683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:06.216964 10679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:06.216951 10681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:06.217272 10668 server_base.cc:1034] running on GCE node
I20260809 08:52:06.217553 10668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:06.217880 10668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:06.219076 10668 hybrid_clock.cc:648] HybridClock initialized: now 1786265526219071 us; error 44 us; skew 500 ppm
Aug 09 08:52:06 dist-test-slave-t3q3 krb5kdc[10647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265526, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:06.221606 10668 init.cc:377] Logged in from keytab as kudu/127.8.154.62@KRBTEST.COM (short username kudu)
I20260809 08:52:06.222147 10668 webserver.cc:460] Webserver started at http://127.8.154.62:43063/ using document root <none> and password file <none>
I20260809 08:52:06.222383 10668 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:06.222453 10668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:06.222615 10668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:06.223652 10668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "9cd9f74df0624cb6a0cacc4027a896fc"
format_stamp: "Formatted at 2026-08-09 08:52:06 on dist-test-slave-t3q3"
server_key: "403a01881841510cf74ad15da7674a5f"
server_key_iv: "351e34450f21f680bcd213a38e60ad29"
server_key_version: "encryptionkey@0"
I20260809 08:52:06.224048 10668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "9cd9f74df0624cb6a0cacc4027a896fc"
format_stamp: "Formatted at 2026-08-09 08:52:06 on dist-test-slave-t3q3"
server_key: "403a01881841510cf74ad15da7674a5f"
server_key_iv: "351e34450f21f680bcd213a38e60ad29"
server_key_version: "encryptionkey@0"
I20260809 08:52:06.225504 10668 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:52:06.226310 10694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:06.226506 10668 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:52:06.226611 10668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "9cd9f74df0624cb6a0cacc4027a896fc"
format_stamp: "Formatted at 2026-08-09 08:52:06 on dist-test-slave-t3q3"
server_key: "403a01881841510cf74ad15da7674a5f"
server_key_iv: "351e34450f21f680bcd213a38e60ad29"
server_key_version: "encryptionkey@0"
I20260809 08:52:06.226699 10668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:06.240285 10668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:06.242745 10668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:06.242905 10668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:06.247584 10668 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:34255
I20260809 08:52:06.247699 10798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:34255 every 8 connection(s)
I20260809 08:52:06.248039 10668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:52:06.248822 10799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:06.249604 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 10668
I20260809 08:52:06.249688 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:52:06.249851 8808 external_mini_cluster.cc:1428] Setting key 6a102ba2326b7b26dd60fb778d4d6075
I20260809 08:52:06.251696 10799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cd9f74df0624cb6a0cacc4027a896fc: Bootstrap starting.
I20260809 08:52:06.252502 10799 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9cd9f74df0624cb6a0cacc4027a896fc: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:06.252806 10799 log.cc:824] T 00000000000000000000000000000000 P 9cd9f74df0624cb6a0cacc4027a896fc: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:06.253561 10799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cd9f74df0624cb6a0cacc4027a896fc: No bootstrap required, opened a new log
I20260809 08:52:06.255532 10799 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9cd9f74df0624cb6a0cacc4027a896fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cd9f74df0624cb6a0cacc4027a896fc" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 34255 } }
I20260809 08:52:06.255736 10799 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9cd9f74df0624cb6a0cacc4027a896fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:06.255776 10799 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9cd9f74df0624cb6a0cacc4027a896fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cd9f74df0624cb6a0cacc4027a896fc, State: Initialized, Role: FOLLOWER
I20260809 08:52:06.255982 10799 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9cd9f74df0624cb6a0cacc4027a896fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cd9f74df0624cb6a0cacc4027a896fc" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 34255 } }
I20260809 08:52:06.256117 10799 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9cd9f74df0624cb6a0cacc4027a896fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:52:06.256156 10799 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9cd9f74df0624cb6a0cacc4027a896fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:52:06.256210 10799 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9cd9f74df0624cb6a0cacc4027a896fc [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:06.256760 10799 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9cd9f74df0624cb6a0cacc4027a896fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cd9f74df0624cb6a0cacc4027a896fc" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 34255 } }
I20260809 08:52:06.256886 10799 leader_election.cc:302] T 00000000000000000000000000000000 P 9cd9f74df0624cb6a0cacc4027a896fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9cd9f74df0624cb6a0cacc4027a896fc; no voters:
I20260809 08:52:06.257095 10799 leader_election.cc:288] T 00000000000000000000000000000000 P 9cd9f74df0624cb6a0cacc4027a896fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:52:06.257153 10804 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9cd9f74df0624cb6a0cacc4027a896fc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:52:06.257346 10804 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9cd9f74df0624cb6a0cacc4027a896fc [term 1 LEADER]: Becoming Leader. State: Replica: 9cd9f74df0624cb6a0cacc4027a896fc, State: Running, Role: LEADER
I20260809 08:52:06.257375 10799 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9cd9f74df0624cb6a0cacc4027a896fc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:52:06.257513 10804 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9cd9f74df0624cb6a0cacc4027a896fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cd9f74df0624cb6a0cacc4027a896fc" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 34255 } }
Aug 09 08:52:06 dist-test-slave-t3q3 krb5kdc[10647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265526, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:06.257959 10804 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cd9f74df0624cb6a0cacc4027a896fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9cd9f74df0624cb6a0cacc4027a896fc. Latest consensus state: current_term: 1 leader_uuid: "9cd9f74df0624cb6a0cacc4027a896fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cd9f74df0624cb6a0cacc4027a896fc" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 34255 } } }
I20260809 08:52:06.257997 10805 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cd9f74df0624cb6a0cacc4027a896fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9cd9f74df0624cb6a0cacc4027a896fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cd9f74df0624cb6a0cacc4027a896fc" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 34255 } } }
I20260809 08:52:06.258180 10805 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cd9f74df0624cb6a0cacc4027a896fc [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:06.258165 10804 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cd9f74df0624cb6a0cacc4027a896fc [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:06.258535 10816 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:52:06.259155 10816 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:52:06.261179 10816 catalog_manager.cc:1347] Generated new cluster ID: 7ee0b60a987e4710828dcf5fa15f6ca6
I20260809 08:52:06.261235 10816 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:52:06.262428 10802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:06.250387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39226 (local address 127.8.154.62:34255)
0809 08:52:06.250583 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:06.250592 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:06.250617 (+ 25us) server_negotiation.cc:407] Connection header received
0809 08:52:06.251023 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:06.251029 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:06.251074 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:06.251252 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:06.251904 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:06.253288 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:06.254085 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:06.255350 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:06.255477 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:06.258370 (+ 2893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:06.258375 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:06.258375 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:06.258386 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:06.260683 (+ 2297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:06.261128 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:06.261132 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:06.261132 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:06.261256 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:06.261499 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:06.261500 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:06.261500 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:06.261742 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:06.261920 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:06.262000 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 08:52:06.262142 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":93,"threads_started":1}
I20260809 08:52:06.286214 10816 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:52:06.286775 10816 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:52:06.294272 10816 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9cd9f74df0624cb6a0cacc4027a896fc: Generated new TSK 0
I20260809 08:52:06.294479 10816 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab.
Entry for principal kudu/127.8.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab.
I20260809 08:52:06.327054 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.1:0
--local_ip_for_outbound_sockets=127.8.154.1
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:34255
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:06.441615 10836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:06.441815 10836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:06.441839 10836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:06.442996 10836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:06.443035 10836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:06.443055 10836 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 08:52:06.443073 10836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.1
I20260809 08:52:06.444576 10836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.154.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.154.62:34255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.1
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:06.444758 10836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:06.444936 10836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:06.450140 10849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:06.450191 10844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:06.450140 10843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:06.450387 10836 server_base.cc:1034] running on GCE node
I20260809 08:52:06.450608 10836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:06.450863 10836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:06.452044 10836 hybrid_clock.cc:648] HybridClock initialized: now 1786265526452019 us; error 38 us; skew 500 ppm
Aug 09 08:52:06 dist-test-slave-t3q3 krb5kdc[10647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265526, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:06.454501 10836 init.cc:377] Logged in from keytab as kudu/127.8.154.1@KRBTEST.COM (short username kudu)
I20260809 08:52:06.454954 10836 webserver.cc:460] Webserver started at http://127.8.154.1:40157/ using document root <none> and password file <none>
I20260809 08:52:06.455183 10836 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:06.455252 10836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:06.455404 10836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:06.456511 10836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-0/data/instance:
uuid: "9ffed0426a0a4f10995bf0e762c9f81f"
format_stamp: "Formatted at 2026-08-09 08:52:06 on dist-test-slave-t3q3"
server_key: "9cc564172ca7607701ba669e28718ede"
server_key_iv: "c582453914c5a5a0f800f587d5e280fb"
server_key_version: "encryptionkey@0"
I20260809 08:52:06.456869 10836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance:
uuid: "9ffed0426a0a4f10995bf0e762c9f81f"
format_stamp: "Formatted at 2026-08-09 08:52:06 on dist-test-slave-t3q3"
server_key: "9cc564172ca7607701ba669e28718ede"
server_key_iv: "c582453914c5a5a0f800f587d5e280fb"
server_key_version: "encryptionkey@0"
I20260809 08:52:06.458351 10836 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:52:06.459067 10861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:06.459322 10836 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:06.459434 10836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-0/wal
uuid: "9ffed0426a0a4f10995bf0e762c9f81f"
format_stamp: "Formatted at 2026-08-09 08:52:06 on dist-test-slave-t3q3"
server_key: "9cc564172ca7607701ba669e28718ede"
server_key_iv: "c582453914c5a5a0f800f587d5e280fb"
server_key_version: "encryptionkey@0"
I20260809 08:52:06.459507 10836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:06.469514 10836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:06.471698 10836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:06.471863 10836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:06.472431 10836 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:06.472469 10836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:06.472491 10836 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:06.472539 10836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:06.479419 10836 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.1:42439
I20260809 08:52:06.479486 11040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.1:42439 every 8 connection(s)
Aug 09 08:52:06 dist-test-slave-t3q3 krb5kdc[10647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265526, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:06.479933 10836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
I20260809 08:52:06.482905 10802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:06.472964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:48345 (local address 127.8.154.62:34255)
0809 08:52:06.473028 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:06.473030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:06.473267 (+ 237us) server_negotiation.cc:407] Connection header received
0809 08:52:06.473742 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:06.473743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:06.473757 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:06.473841 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:06.474644 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:06.475504 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:06.477104 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:06.477476 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:06.477564 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:06.480261 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:06.480262 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:06.480263 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:06.480270 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:06.481747 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:06.482100 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:06.482101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:06.482101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:06.482150 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:06.482379 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:06.482379 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:06.482380 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:06.482480 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:06.482530 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:06.482766 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 08:52:06.482842 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 08:52:06.483043 10873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:06.472972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:34255 (local address 127.8.154.1:48345)
0809 08:52:06.473134 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:06.473169 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:06.473620 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:06.473891 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:06.473894 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:06.474258 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:06.474534 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:06.474540 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:06.475608 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:06.475609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:06.477005 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:06.477008 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:06.477597 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:06.477598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:06.477712 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:06.478201 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:06.478212 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:06.480138 (+ 1926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:06.481818 (+ 1680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:06.481823 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:06.481825 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:06.482036 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:06.482225 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:06.482225 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:06.482226 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:06.482324 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:06.482535 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:06.482539 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:06.482642 (+ 103us) client_negotiation.cc:769] Sending connection context
0809 08:52:06.482732 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 08:52:06.482851 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":88,"threads_started":1}
I20260809 08:52:06.483481 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 10836
I20260809 08:52:06.483585 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance
I20260809 08:52:06.483654 11041 heartbeater.cc:347] Connected to a master server at 127.8.154.62:34255
I20260809 08:52:06.483757 11041 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:06.483765 8808 external_mini_cluster.cc:1428] Setting key b6ef4e3d068d4a5d2b904cb4025ba4f4
I20260809 08:52:06.483946 11041 heartbeater.cc:511] Master 127.8.154.62:34255 requested a full tablet report, sending...
I20260809 08:52:06.484493 10717 ts_manager.cc:194] Registered new tserver with Master: 9ffed0426a0a4f10995bf0e762c9f81f (127.8.154.1:42439)
I20260809 08:52:06.485102 10717 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.1@KRBTEST.COM'} at 127.8.154.1:48345
WARNING: no policy specified for kudu/127.8.154.2@KRBTEST.COM; defaulting to no policy
I20260809 08:52:06.494155 10802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:06.485911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:51329 (local address 127.8.154.62:34255)
0809 08:52:06.485948 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:06.485950 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:06.488279 (+ 2329us) server_negotiation.cc:407] Connection header received
0809 08:52:06.488312 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:06.488313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:06.488326 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:06.488403 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:06.488685 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:06.489469 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:06.490143 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:06.490477 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:06.490556 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:06.491422 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:06.491423 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:06.491424 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:06.491433 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:06.493021 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:06.493397 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:06.493399 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:06.493399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:06.493461 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:06.493689 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:06.493690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:06.493690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:06.493806 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:06.493866 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:06.494040 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 08:52:06.494096 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:52:06.494226 11051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:06.485848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:34255 (local address 127.8.154.1:51329)
0809 08:52:06.488177 (+ 2329us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:06.488188 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:06.488249 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:06.488404 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:06.488404 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:06.488563 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:06.488610 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:06.488611 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:06.489577 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:06.489578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:06.490058 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:06.490059 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:06.490560 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:06.490561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:06.490618 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:06.491054 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:06.491058 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:06.491328 (+ 270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:06.493106 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:06.493107 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:06.493107 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:06.493287 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:06.493539 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:06.493540 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:06.493540 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:06.493610 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:06.493906 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:06.493906 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:06.493949 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 08:52:06.494020 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 08:52:06.494118 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2300,"thread_start_us":68,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab.
Entry for principal kudu/127.8.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab.
I20260809 08:52:06.510558 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.2:0
--local_ip_for_outbound_sockets=127.8.154.2
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:34255
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:06.594275 11054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:06.594484 11054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:06.594508 11054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:06.595600 11054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:06.595636 11054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:06.595656 11054 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 08:52:06.595674 11054 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.2
I20260809 08:52:06.597191 11054 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.154.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.154.62:34255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.2
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:06.597396 11054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:06.597579 11054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:06.602600 11062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:06.602602 11066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:06.602949 11054 server_base.cc:1034] running on GCE node
W20260809 08:52:06.602662 11063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:06.603279 11054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:06.603543 11054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:06.604710 11054 hybrid_clock.cc:648] HybridClock initialized: now 1786265526604695 us; error 44 us; skew 500 ppm
Aug 09 08:52:06 dist-test-slave-t3q3 krb5kdc[10647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265526, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:06.607197 11054 init.cc:377] Logged in from keytab as kudu/127.8.154.2@KRBTEST.COM (short username kudu)
I20260809 08:52:06.607722 11054 webserver.cc:460] Webserver started at http://127.8.154.2:40975/ using document root <none> and password file <none>
I20260809 08:52:06.607913 11054 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:06.607954 11054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:06.608047 11054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:06.609122 11054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-1/data/instance:
uuid: "768011a2337d4d49b276f2bae87ac0ca"
format_stamp: "Formatted at 2026-08-09 08:52:06 on dist-test-slave-t3q3"
server_key: "008e18442ab2f9844b8241f94392121d"
server_key_iv: "153e73bef4ac8f0dc6330600013af465"
server_key_version: "encryptionkey@0"
I20260809 08:52:06.609498 11054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance:
uuid: "768011a2337d4d49b276f2bae87ac0ca"
format_stamp: "Formatted at 2026-08-09 08:52:06 on dist-test-slave-t3q3"
server_key: "008e18442ab2f9844b8241f94392121d"
server_key_iv: "153e73bef4ac8f0dc6330600013af465"
server_key_version: "encryptionkey@0"
I20260809 08:52:06.610816 11054 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:52:06.611500 11077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:06.611701 11054 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:52:06.611800 11054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-1/wal
uuid: "768011a2337d4d49b276f2bae87ac0ca"
format_stamp: "Formatted at 2026-08-09 08:52:06 on dist-test-slave-t3q3"
server_key: "008e18442ab2f9844b8241f94392121d"
server_key_iv: "153e73bef4ac8f0dc6330600013af465"
server_key_version: "encryptionkey@0"
I20260809 08:52:06.611877 11054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:06.625057 11054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:06.626935 11054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:06.627104 11054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:06.627660 11054 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:06.627701 11054 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:06.627723 11054 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:06.627738 11054 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 08:52:06 dist-test-slave-t3q3 krb5kdc[10647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265526, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:06.635747 11054 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.2:38853
I20260809 08:52:06.636160 11054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
I20260809 08:52:06.636202 11285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.2:38853 every 8 connection(s)
I20260809 08:52:06.638414 10802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:06.628056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:42185 (local address 127.8.154.62:34255)
0809 08:52:06.628125 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:06.628127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:06.628346 (+ 219us) server_negotiation.cc:407] Connection header received
0809 08:52:06.628900 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:06.628901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:06.628916 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:06.628978 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:06.629695 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:06.630574 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:06.631644 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:06.632020 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:06.632131 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:06.635718 (+ 3587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:06.635720 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:06.635720 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:06.635728 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:06.637292 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:06.637653 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:06.637653 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:06.637654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:06.637707 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:06.637902 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:06.637902 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:06.637902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:06.638005 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:06.638056 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:06.638288 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 08:52:06.638351 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 08:52:06.638559 11104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:06.628082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:34255 (local address 127.8.154.2:42185)
0809 08:52:06.628227 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:06.628261 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:06.628765 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:06.629085 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:06.629090 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:06.629397 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:06.629579 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:06.629584 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:06.630711 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:06.630713 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:06.631554 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:06.631556 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:06.632137 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:06.632138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:06.632292 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:06.632821 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:06.632832 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:06.635611 (+ 2779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:06.637363 (+ 1752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:06.637369 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:06.637371 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:06.637584 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:06.637766 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:06.637766 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:06.637767 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:06.637851 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:06.638062 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:06.638067 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:06.638180 (+ 113us) client_negotiation.cc:769] Sending connection context
0809 08:52:06.638260 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 08:52:06.638375 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":80,"threads_started":1}
I20260809 08:52:06.639036 11287 heartbeater.cc:347] Connected to a master server at 127.8.154.62:34255
I20260809 08:52:06.639117 11287 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:06.639317 11287 heartbeater.cc:511] Master 127.8.154.62:34255 requested a full tablet report, sending...
I20260809 08:52:06.639766 10720 ts_manager.cc:194] Registered new tserver with Master: 768011a2337d4d49b276f2bae87ac0ca (127.8.154.2:38853)
I20260809 08:52:06.640211 10720 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.2@KRBTEST.COM'} at 127.8.154.2:42185
I20260809 08:52:06.645207 10802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:06.640302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:53043 (local address 127.8.154.62:34255)
0809 08:52:06.640343 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:06.640344 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:06.640380 (+ 36us) server_negotiation.cc:407] Connection header received
0809 08:52:06.640443 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:06.640443 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:06.640454 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:06.640500 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:06.640831 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:06.641357 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:06.642009 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:06.642197 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:06.642250 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:06.643001 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:06.643002 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:06.643002 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:06.643008 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:06.644279 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:06.644578 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:06.644578 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:06.644578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:06.644628 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:06.644796 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:06.644796 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:06.644797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:06.644921 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:06.644973 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:06.645081 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 08:52:06.645139 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 08:52:06.645210 11298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:06.640170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:34255 (local address 127.8.154.2:53043)
0809 08:52:06.640319 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:06.640331 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:06.640395 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:06.640504 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:06.640505 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:06.640693 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:06.640756 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:06.640757 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:06.641441 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:06.641442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:06.641941 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:06.641942 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:06.642264 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:06.642265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:06.642326 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:06.642651 (+ 325us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:06.642655 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:06.642926 (+ 271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:06.644350 (+ 1424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:06.644351 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:06.644352 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:06.644516 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:06.644684 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:06.644685 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:06.644685 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:06.644746 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:06.644977 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:06.644977 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:06.645014 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 08:52:06.645064 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 08:52:06.645140 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":78,"threads_started":1}
I20260809 08:52:06.645803 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 11054
I20260809 08:52:06.645889 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance
I20260809 08:52:06.646073 8808 external_mini_cluster.cc:1428] Setting key 2aa4326e0098d3ae61a86bd369b83837
WARNING: no policy specified for kudu/127.8.154.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab.
Entry for principal kudu/127.8.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab.
I20260809 08:52:06.672019 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.3:0
--local_ip_for_outbound_sockets=127.8.154.3
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:34255
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:40193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:06.760792 11301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:06.761001 11301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:06.761026 11301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:06.762264 11301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:06.762385 11301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:06.762436 11301 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 08:52:06.762468 11301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.3
I20260809 08:52:06.764020 11301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:40193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.154.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.154.62:34255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.3
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:06.764328 11301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:06.764575 11301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:06.769961 11312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:06.769979 11311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:06.769979 11301 server_base.cc:1034] running on GCE node
W20260809 08:52:06.769963 11316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:06.770473 11301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:06.770746 11301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:06.771893 11301 hybrid_clock.cc:648] HybridClock initialized: now 1786265526771883 us; error 30 us; skew 500 ppm
Aug 09 08:52:06 dist-test-slave-t3q3 krb5kdc[10647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265526, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:06.774497 11301 init.cc:377] Logged in from keytab as kudu/127.8.154.3@KRBTEST.COM (short username kudu)
I20260809 08:52:06.774966 11301 webserver.cc:460] Webserver started at http://127.8.154.3:45837/ using document root <none> and password file <none>
I20260809 08:52:06.775233 11301 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:06.775307 11301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:06.775462 11301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:06.776624 11301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-2/data/instance:
uuid: "b50224ba75a44c32a84878472b542820"
format_stamp: "Formatted at 2026-08-09 08:52:06 on dist-test-slave-t3q3"
server_key: "d4650f4724470cf803cc5c700516a3a5"
server_key_iv: "c73f46cd747792b0e4b173758db9b3db"
server_key_version: "encryptionkey@0"
I20260809 08:52:06.777055 11301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance:
uuid: "b50224ba75a44c32a84878472b542820"
format_stamp: "Formatted at 2026-08-09 08:52:06 on dist-test-slave-t3q3"
server_key: "d4650f4724470cf803cc5c700516a3a5"
server_key_iv: "c73f46cd747792b0e4b173758db9b3db"
server_key_version: "encryptionkey@0"
I20260809 08:52:06.778491 11301 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:52:06.779225 11329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:06.779415 11301 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:52:06.779526 11301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-2/wal
uuid: "b50224ba75a44c32a84878472b542820"
format_stamp: "Formatted at 2026-08-09 08:52:06 on dist-test-slave-t3q3"
server_key: "d4650f4724470cf803cc5c700516a3a5"
server_key_iv: "c73f46cd747792b0e4b173758db9b3db"
server_key_version: "encryptionkey@0"
I20260809 08:52:06.779603 11301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:06.798620 11301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:06.800907 11301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:06.801059 11301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:06.801620 11301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:06.801688 11301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:06.801724 11301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:06.801755 11301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 08:52:06 dist-test-slave-t3q3 krb5kdc[10647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265526, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:06.809662 11301 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.3:45107
I20260809 08:52:06.809741 11509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.3:45107 every 8 connection(s)
I20260809 08:52:06.810122 11301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
I20260809 08:52:06.812265 10802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:06.802117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:51227 (local address 127.8.154.62:34255)
0809 08:52:06.802193 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:06.802196 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:06.802383 (+ 187us) server_negotiation.cc:407] Connection header received
0809 08:52:06.802857 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:06.802859 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:06.802871 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:06.802961 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:06.803845 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:06.804744 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:06.806380 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:06.806738 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:06.806823 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:06.809463 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:06.809465 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:06.809465 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:06.809474 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:06.811012 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:06.811419 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:06.811420 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:06.811421 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:06.811475 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:06.811684 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:06.811685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:06.811685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:06.811787 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:06.811834 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:06.812118 (+ 284us) server_negotiation.cc:299] Negotiation successful
0809 08:52:06.812197 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 08:52:06.812471 11346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:06.802092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:34255 (local address 127.8.154.3:51227)
0809 08:52:06.802243 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:06.802283 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:06.802763 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:06.803013 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:06.803041 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:06.803420 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:06.803724 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:06.803733 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:06.804852 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:06.804854 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:06.806280 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:06.806284 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:06.806841 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:06.806842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:06.806995 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:06.807572 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:06.807581 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:06.809336 (+ 1755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:06.811087 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:06.811091 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:06.811093 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:06.811365 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:06.811535 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:06.811536 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:06.811536 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:06.811634 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:06.811839 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:06.811843 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:06.811955 (+ 112us) client_negotiation.cc:769] Sending connection context
0809 08:52:06.812039 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 08:52:06.812232 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":82,"threads_started":1}
I20260809 08:52:06.812983 11510 heartbeater.cc:347] Connected to a master server at 127.8.154.62:34255
I20260809 08:52:06.813104 11510 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:06.813298 11510 heartbeater.cc:511] Master 127.8.154.62:34255 requested a full tablet report, sending...
I20260809 08:52:06.813829 10717 ts_manager.cc:194] Registered new tserver with Master: b50224ba75a44c32a84878472b542820 (127.8.154.3:45107)
I20260809 08:52:06.814307 10717 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.3@KRBTEST.COM'} at 127.8.154.3:51227
I20260809 08:52:06.817667 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 11301
I20260809 08:52:06.817762 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance
I20260809 08:52:06.817950 8808 external_mini_cluster.cc:1428] Setting key fe4f256d0e6d26d229e6765a2f3c898f
I20260809 08:52:06.818800 8808 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:52:06.819948 10802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:06.814110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:33285 (local address 127.8.154.62:34255)
0809 08:52:06.814164 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:06.814166 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:06.814228 (+ 62us) server_negotiation.cc:407] Connection header received
0809 08:52:06.814302 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:06.814303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:06.814312 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:06.814367 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:06.814760 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:06.815288 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:06.816114 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:06.816307 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:06.816394 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:06.817305 (+ 911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:06.817306 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:06.817307 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:06.817313 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:06.818867 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:06.819230 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:06.819231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:06.819231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:06.819292 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:06.819506 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:06.819507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:06.819507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:06.819609 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:06.819663 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:06.819830 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 08:52:06.819891 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 08:52:06.820120 11517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:06.814036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:34255 (local address 127.8.154.3:33285)
0809 08:52:06.814173 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:06.814184 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:06.814245 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:06.814372 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:06.814373 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:06.814584 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:06.814685 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:06.814687 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:06.815390 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:06.815391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:06.815989 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:06.815991 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:06.816397 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:06.816398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:06.816473 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:06.816918 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:06.816923 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:06.817199 (+ 276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:06.818955 (+ 1756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:06.818956 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:06.818956 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:06.819149 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:06.819369 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:06.819370 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:06.819370 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:06.819440 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:06.819677 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:06.819678 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:06.819728 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 08:52:06.819798 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 08:52:06.819923 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":79,"threads_started":1}
Aug 09 08:52:06 dist-test-slave-t3q3 krb5kdc[10647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265526, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 08:52:06 dist-test-slave-t3q3 krb5kdc[10647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265526, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:06.842633 10802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:06.836135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39232 (local address 127.8.154.62:34255)
0809 08:52:06.836193 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:06.836194 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:06.836201 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:52:06.836224 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:06.836224 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:06.836233 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:06.836276 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:06.836841 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:06.837337 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:06.838026 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:06.838207 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:06.838257 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:06.840446 (+ 2189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:06.840448 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:06.840448 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:06.840461 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:06.841620 (+ 1159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:06.841986 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:06.841987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:06.841988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:06.842047 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:06.842234 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:06.842234 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:06.842235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:06.842340 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:06.842411 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:06.842506 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 08:52:06.842569 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 08:52:06.844310 10717 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39232:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:52:06.845947 10717 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:52:06.852475 11536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:06.848623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54248 (local address 127.8.154.1:42439)
0809 08:52:06.848810 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:06.848816 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:06.848828 (+ 12us) server_negotiation.cc:407] Connection header received
0809 08:52:06.848875 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:06.848877 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:06.848895 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:06.849000 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:06.849162 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:06.849981 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:06.851366 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:06.852016 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:06.852153 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:06.852229 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:06.852344 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 08:52:06.852382 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":115,"threads_started":1}
I20260809 08:52:06.852468 11533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:06.848422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:42439 (local address 127.0.0.1:54248)
0809 08:52:06.848579 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:06.848625 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:06.848697 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:06.849008 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:06.849010 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:06.849012 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:06.849078 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:06.849080 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:06.850074 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:06.850075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:06.851269 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:06.851272 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:06.852157 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:06.852158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:06.852240 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:06.852253 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 08:52:06.852329 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 08:52:06.852382 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":47,"threads_started":1}
I20260809 08:52:06.852823 11534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:06.848447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:38853 (local address 127.0.0.1:38236)
0809 08:52:06.849180 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:06.849192 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:06.849244 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:06.849496 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:06.849497 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:06.849498 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:06.849545 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:06.849546 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:06.850740 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:06.850742 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:06.851657 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:06.851659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:06.852562 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:06.852563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:06.852653 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:06.852658 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:52:06.852705 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 08:52:06.852755 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":711,"thread_start_us":98,"threads_started":1}
I20260809 08:52:06.853096 11538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:06.848937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38236 (local address 127.8.154.2:38853)
0809 08:52:06.849124 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:06.849128 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:06.849304 (+ 176us) server_negotiation.cc:407] Connection header received
0809 08:52:06.849347 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:06.849350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:06.849366 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:06.849434 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:06.849621 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:06.850648 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:06.851752 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:06.852405 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:06.852475 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:06.852522 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:06.852945 (+ 423us) server_negotiation.cc:299] Negotiation successful
0809 08:52:06.853001 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":78,"threads_started":1}
I20260809 08:52:06.853257 10942 tablet_service.cc:1467] Processing CreateTablet for tablet 88f5fff66a3442af8de842b54a80359e (DEFAULT_TABLE table=test-table [id=fe04d1e98fed43d69bef2a5065651c6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:06.853679 10942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88f5fff66a3442af8de842b54a80359e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:06.853705 11178 tablet_service.cc:1467] Processing CreateTablet for tablet 88f5fff66a3442af8de842b54a80359e (DEFAULT_TABLE table=test-table [id=fe04d1e98fed43d69bef2a5065651c6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:06.854087 11178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88f5fff66a3442af8de842b54a80359e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:06.856498 11532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:06.848422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:45107 (local address 127.0.0.1:51504)
0809 08:52:06.848829 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:06.848841 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:06.848893 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:06.849151 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:06.849152 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:06.849155 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:06.849216 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:06.849217 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:06.850544 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:06.850545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:06.851637 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:06.851639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:06.856158 (+ 4519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:06.856159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:06.856241 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:06.856246 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:52:06.856341 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 08:52:06.856396 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"mutex_wait_us":27,"thread_start_us":91,"threads_started":1}
I20260809 08:52:06.856842 11537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:06.848713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51504 (local address 127.8.154.3:45107)
0809 08:52:06.848888 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:06.848893 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:06.848906 (+ 13us) server_negotiation.cc:407] Connection header received
0809 08:52:06.848954 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:06.848956 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:06.848974 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:06.849054 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:06.849315 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:06.850439 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:06.852161 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:06.853151 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:06.853226 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:06.853290 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:06.856676 (+ 3386us) server_negotiation.cc:299] Negotiation successful
0809 08:52:06.856723 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":105,"threads_started":1}
I20260809 08:52:06.856823 11540 tablet_bootstrap.cc:492] T 88f5fff66a3442af8de842b54a80359e P 9ffed0426a0a4f10995bf0e762c9f81f: Bootstrap starting.
I20260809 08:52:06.857182 11541 tablet_bootstrap.cc:492] T 88f5fff66a3442af8de842b54a80359e P 768011a2337d4d49b276f2bae87ac0ca: Bootstrap starting.
I20260809 08:52:06.857472 11409 tablet_service.cc:1467] Processing CreateTablet for tablet 88f5fff66a3442af8de842b54a80359e (DEFAULT_TABLE table=test-table [id=fe04d1e98fed43d69bef2a5065651c6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:06.857822 11409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88f5fff66a3442af8de842b54a80359e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:06.857895 11541 tablet_bootstrap.cc:654] T 88f5fff66a3442af8de842b54a80359e P 768011a2337d4d49b276f2bae87ac0ca: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:06.858352 11541 log.cc:824] T 88f5fff66a3442af8de842b54a80359e P 768011a2337d4d49b276f2bae87ac0ca: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:06.859176 11541 tablet_bootstrap.cc:492] T 88f5fff66a3442af8de842b54a80359e P 768011a2337d4d49b276f2bae87ac0ca: No bootstrap required, opened a new log
I20260809 08:52:06.859253 11541 ts_tablet_manager.cc:1397] T 88f5fff66a3442af8de842b54a80359e P 768011a2337d4d49b276f2bae87ac0ca: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:52:06.860800 11540 tablet_bootstrap.cc:654] T 88f5fff66a3442af8de842b54a80359e P 9ffed0426a0a4f10995bf0e762c9f81f: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:06.861125 11540 log.cc:824] T 88f5fff66a3442af8de842b54a80359e P 9ffed0426a0a4f10995bf0e762c9f81f: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:06.861397 11541 raft_consensus.cc:348] T 88f5fff66a3442af8de842b54a80359e P 768011a2337d4d49b276f2bae87ac0ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50224ba75a44c32a84878472b542820" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45107 } } peers { permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 } } peers { permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 } }
I20260809 08:52:06.861579 11541 raft_consensus.cc:374] T 88f5fff66a3442af8de842b54a80359e P 768011a2337d4d49b276f2bae87ac0ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:06.861629 11541 raft_consensus.cc:729] T 88f5fff66a3442af8de842b54a80359e P 768011a2337d4d49b276f2bae87ac0ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 768011a2337d4d49b276f2bae87ac0ca, State: Initialized, Role: FOLLOWER
I20260809 08:52:06.861758 11541 consensus_queue.cc:260] T 88f5fff66a3442af8de842b54a80359e P 768011a2337d4d49b276f2bae87ac0ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50224ba75a44c32a84878472b542820" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45107 } } peers { permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 } } peers { permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 } }
I20260809 08:52:06.862062 11540 tablet_bootstrap.cc:492] T 88f5fff66a3442af8de842b54a80359e P 9ffed0426a0a4f10995bf0e762c9f81f: No bootstrap required, opened a new log
I20260809 08:52:06.862066 11541 ts_tablet_manager.cc:1428] T 88f5fff66a3442af8de842b54a80359e P 768011a2337d4d49b276f2bae87ac0ca: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 08:52:06.862083 11287 heartbeater.cc:503] Master 127.8.154.62:34255 was elected leader, sending a full tablet report...
I20260809 08:52:06.862138 11540 ts_tablet_manager.cc:1397] T 88f5fff66a3442af8de842b54a80359e P 9ffed0426a0a4f10995bf0e762c9f81f: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260809 08:52:06.862238 11546 tablet_bootstrap.cc:492] T 88f5fff66a3442af8de842b54a80359e P b50224ba75a44c32a84878472b542820: Bootstrap starting.
I20260809 08:52:06.863148 11546 tablet_bootstrap.cc:654] T 88f5fff66a3442af8de842b54a80359e P b50224ba75a44c32a84878472b542820: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:06.863504 11546 log.cc:824] T 88f5fff66a3442af8de842b54a80359e P b50224ba75a44c32a84878472b542820: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:06.864204 11546 tablet_bootstrap.cc:492] T 88f5fff66a3442af8de842b54a80359e P b50224ba75a44c32a84878472b542820: No bootstrap required, opened a new log
I20260809 08:52:06.864260 11546 ts_tablet_manager.cc:1397] T 88f5fff66a3442af8de842b54a80359e P b50224ba75a44c32a84878472b542820: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:52:06.864143 11540 raft_consensus.cc:348] T 88f5fff66a3442af8de842b54a80359e P 9ffed0426a0a4f10995bf0e762c9f81f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50224ba75a44c32a84878472b542820" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45107 } } peers { permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 } } peers { permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 } }
I20260809 08:52:06.864328 11540 raft_consensus.cc:374] T 88f5fff66a3442af8de842b54a80359e P 9ffed0426a0a4f10995bf0e762c9f81f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:06.864370 11540 raft_consensus.cc:729] T 88f5fff66a3442af8de842b54a80359e P 9ffed0426a0a4f10995bf0e762c9f81f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ffed0426a0a4f10995bf0e762c9f81f, State: Initialized, Role: FOLLOWER
I20260809 08:52:06.864503 11540 consensus_queue.cc:260] T 88f5fff66a3442af8de842b54a80359e P 9ffed0426a0a4f10995bf0e762c9f81f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50224ba75a44c32a84878472b542820" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45107 } } peers { permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 } } peers { permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 } }
I20260809 08:52:06.864815 11540 ts_tablet_manager.cc:1428] T 88f5fff66a3442af8de842b54a80359e P 9ffed0426a0a4f10995bf0e762c9f81f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:52:06.864856 11041 heartbeater.cc:503] Master 127.8.154.62:34255 was elected leader, sending a full tablet report...
I20260809 08:52:06.866166 11546 raft_consensus.cc:348] T 88f5fff66a3442af8de842b54a80359e P b50224ba75a44c32a84878472b542820 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50224ba75a44c32a84878472b542820" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45107 } } peers { permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 } } peers { permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 } }
I20260809 08:52:06.866285 11546 raft_consensus.cc:374] T 88f5fff66a3442af8de842b54a80359e P b50224ba75a44c32a84878472b542820 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:06.866305 11546 raft_consensus.cc:729] T 88f5fff66a3442af8de842b54a80359e P b50224ba75a44c32a84878472b542820 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b50224ba75a44c32a84878472b542820, State: Initialized, Role: FOLLOWER
I20260809 08:52:06.866379 11546 consensus_queue.cc:260] T 88f5fff66a3442af8de842b54a80359e P b50224ba75a44c32a84878472b542820 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50224ba75a44c32a84878472b542820" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45107 } } peers { permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 } } peers { permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 } }
I20260809 08:52:06.866571 11546 ts_tablet_manager.cc:1428] T 88f5fff66a3442af8de842b54a80359e P b50224ba75a44c32a84878472b542820: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:52:06.866647 11510 heartbeater.cc:503] Master 127.8.154.62:34255 was elected leader, sending a full tablet report...
W20260809 08:52:06.887054 11288 tablet.cc:2369] T 88f5fff66a3442af8de842b54a80359e P 768011a2337d4d49b276f2bae87ac0ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 08:52:06.980899 11042 tablet.cc:2369] T 88f5fff66a3442af8de842b54a80359e P 9ffed0426a0a4f10995bf0e762c9f81f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:52:07.014124 11555 raft_consensus.cc:482] T 88f5fff66a3442af8de842b54a80359e P b50224ba75a44c32a84878472b542820 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:52:07.014300 11555 raft_consensus.cc:504] T 88f5fff66a3442af8de842b54a80359e P b50224ba75a44c32a84878472b542820 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50224ba75a44c32a84878472b542820" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45107 } } peers { permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 } } peers { permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 } }
I20260809 08:52:07.014707 11555 leader_election.cc:288] T 88f5fff66a3442af8de842b54a80359e P b50224ba75a44c32a84878472b542820 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9ffed0426a0a4f10995bf0e762c9f81f (127.8.154.1:42439), 768011a2337d4d49b276f2bae87ac0ca (127.8.154.2:38853)
I20260809 08:52:07.017628 11346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:07.014892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:42439 (local address 127.8.154.3:41951)
0809 08:52:07.014941 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:07.014954 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:07.015007 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:07.015213 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:07.015215 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:07.015217 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:07.015298 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:07.015300 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:07.015970 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:07.015971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:07.016787 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:07.016789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:07.017345 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:07.017346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:07.017440 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:07.017443 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:52:07.017512 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 08:52:07.017556 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260809 08:52:07.017628 11536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:07.014967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:41951 (local address 127.8.154.1:42439)
0809 08:52:07.015038 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:07.015040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:07.015052 (+ 12us) server_negotiation.cc:407] Connection header received
0809 08:52:07.015086 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:07.015087 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:07.015100 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:07.015172 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:07.015389 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:07.015879 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:07.016884 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:07.017257 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:07.017345 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:07.017356 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:07.017520 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 08:52:07.017560 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 08:52:07.017745 11538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:07.015056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:57113 (local address 127.8.154.2:38853)
0809 08:52:07.015107 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:07.015109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:07.015221 (+ 112us) server_negotiation.cc:407] Connection header received
0809 08:52:07.015318 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:07.015319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:07.015330 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:07.015384 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:07.015565 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:07.016038 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:07.017013 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:07.017372 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:07.017467 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:07.017477 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:07.017643 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 08:52:07.017680 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 08:52:07.017767 11558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:07.015003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:38853 (local address 127.8.154.3:57113)
0809 08:52:07.015162 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:07.015172 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:07.015245 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:07.015407 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:07.015408 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:07.015410 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:07.015484 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:07.015486 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:07.016175 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:07.016177 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:07.016905 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:07.016906 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:07.017474 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:07.017475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:07.017565 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:07.017568 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:52:07.017637 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 08:52:07.017678 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":102,"threads_started":1}
I20260809 08:52:07.018164 11210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88f5fff66a3442af8de842b54a80359e" candidate_uuid: "b50224ba75a44c32a84878472b542820" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "768011a2337d4d49b276f2bae87ac0ca" is_pre_election: true
I20260809 08:52:07.018219 10977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88f5fff66a3442af8de842b54a80359e" candidate_uuid: "b50224ba75a44c32a84878472b542820" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" is_pre_election: true
I20260809 08:52:07.018329 11210 raft_consensus.cc:2454] T 88f5fff66a3442af8de842b54a80359e P 768011a2337d4d49b276f2bae87ac0ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b50224ba75a44c32a84878472b542820 in term 0.
I20260809 08:52:07.018366 10977 raft_consensus.cc:2454] T 88f5fff66a3442af8de842b54a80359e P 9ffed0426a0a4f10995bf0e762c9f81f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b50224ba75a44c32a84878472b542820 in term 0.
I20260809 08:52:07.018574 11330 leader_election.cc:302] T 88f5fff66a3442af8de842b54a80359e P b50224ba75a44c32a84878472b542820 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 768011a2337d4d49b276f2bae87ac0ca, b50224ba75a44c32a84878472b542820; no voters:
I20260809 08:52:07.018707 11555 raft_consensus.cc:2781] T 88f5fff66a3442af8de842b54a80359e P b50224ba75a44c32a84878472b542820 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:52:07.018759 11555 raft_consensus.cc:482] T 88f5fff66a3442af8de842b54a80359e P b50224ba75a44c32a84878472b542820 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:52:07.018787 11555 raft_consensus.cc:3037] T 88f5fff66a3442af8de842b54a80359e P b50224ba75a44c32a84878472b542820 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:07.019486 11555 raft_consensus.cc:504] T 88f5fff66a3442af8de842b54a80359e P b50224ba75a44c32a84878472b542820 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50224ba75a44c32a84878472b542820" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45107 } } peers { permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 } } peers { permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 } }
I20260809 08:52:07.019621 11555 leader_election.cc:288] T 88f5fff66a3442af8de842b54a80359e P b50224ba75a44c32a84878472b542820 [CANDIDATE]: Term 1 election: Requested vote from peers 9ffed0426a0a4f10995bf0e762c9f81f (127.8.154.1:42439), 768011a2337d4d49b276f2bae87ac0ca (127.8.154.2:38853)
I20260809 08:52:07.019760 10977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88f5fff66a3442af8de842b54a80359e" candidate_uuid: "b50224ba75a44c32a84878472b542820" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ffed0426a0a4f10995bf0e762c9f81f"
I20260809 08:52:07.019810 11210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88f5fff66a3442af8de842b54a80359e" candidate_uuid: "b50224ba75a44c32a84878472b542820" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "768011a2337d4d49b276f2bae87ac0ca"
I20260809 08:52:07.019857 10977 raft_consensus.cc:3037] T 88f5fff66a3442af8de842b54a80359e P 9ffed0426a0a4f10995bf0e762c9f81f [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:07.019897 11210 raft_consensus.cc:3037] T 88f5fff66a3442af8de842b54a80359e P 768011a2337d4d49b276f2bae87ac0ca [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:07.020645 10977 raft_consensus.cc:2454] T 88f5fff66a3442af8de842b54a80359e P 9ffed0426a0a4f10995bf0e762c9f81f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b50224ba75a44c32a84878472b542820 in term 1.
I20260809 08:52:07.020682 11210 raft_consensus.cc:2454] T 88f5fff66a3442af8de842b54a80359e P 768011a2337d4d49b276f2bae87ac0ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b50224ba75a44c32a84878472b542820 in term 1.
I20260809 08:52:07.021294 11334 leader_election.cc:302] T 88f5fff66a3442af8de842b54a80359e P b50224ba75a44c32a84878472b542820 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 768011a2337d4d49b276f2bae87ac0ca, 9ffed0426a0a4f10995bf0e762c9f81f, b50224ba75a44c32a84878472b542820; no voters:
I20260809 08:52:07.021411 11555 raft_consensus.cc:2781] T 88f5fff66a3442af8de842b54a80359e P b50224ba75a44c32a84878472b542820 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:52:07.021564 11555 raft_consensus.cc:686] T 88f5fff66a3442af8de842b54a80359e P b50224ba75a44c32a84878472b542820 [term 1 LEADER]: Becoming Leader. State: Replica: b50224ba75a44c32a84878472b542820, State: Running, Role: LEADER
I20260809 08:52:07.021667 11555 consensus_queue.cc:237] T 88f5fff66a3442af8de842b54a80359e P b50224ba75a44c32a84878472b542820 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50224ba75a44c32a84878472b542820" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45107 } } peers { permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 } } peers { permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 } }
I20260809 08:52:07.022562 10717 catalog_manager.cc:5495] T 88f5fff66a3442af8de842b54a80359e P b50224ba75a44c32a84878472b542820 reported cstate change: term changed from 0 to 1, leader changed from <none> to b50224ba75a44c32a84878472b542820 (127.8.154.3). New cstate: current_term: 1 leader_uuid: "b50224ba75a44c32a84878472b542820" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b50224ba75a44c32a84878472b542820" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 } health_report { overall_health: UNKNOWN } } }
I20260809 08:52:07.035024 11537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:07.032655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51506 (local address 127.8.154.3:45107)
0809 08:52:07.032708 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:07.032710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:07.032720 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:52:07.032813 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:07.032814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:07.032830 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:07.032898 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:52:07.033105 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:07.033555 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:07.034284 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:07.034457 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:07.034508 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:07.034687 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:52:07.034819 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:52:07.034905 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:07.034956 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 08:52:07.034975 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 08:52:07.036962 10977 raft_consensus.cc:1261] T 88f5fff66a3442af8de842b54a80359e P 9ffed0426a0a4f10995bf0e762c9f81f [term 1 FOLLOWER]: Refusing update from remote peer b50224ba75a44c32a84878472b542820: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:52:07.036965 11210 raft_consensus.cc:1261] T 88f5fff66a3442af8de842b54a80359e P 768011a2337d4d49b276f2bae87ac0ca [term 1 FOLLOWER]: Refusing update from remote peer b50224ba75a44c32a84878472b542820: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:52:07.037321 11559 consensus_queue.cc:1035] T 88f5fff66a3442af8de842b54a80359e P b50224ba75a44c32a84878472b542820 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:52:07.037448 11555 consensus_queue.cc:1035] T 88f5fff66a3442af8de842b54a80359e P b50224ba75a44c32a84878472b542820 [LEADER]: Connected to new peer: Peer: permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:52:07.040663 11566 mvcc.cc:205] Tried to move back new op lower bound from 7316543598740606976 to 7316543598681563136. Current Snapshot: MvccSnapshot[applied={T|T < 7316543598740606976}]
I20260809 08:52:07.044147 10720 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:52:07.044298 10720 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:52:07.045198 10720 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9cd9f74df0624cb6a0cacc4027a896fc: Sending DeleteTablet for 3 replicas of tablet 88f5fff66a3442af8de842b54a80359e
I20260809 08:52:07.045637 11409 tablet_service.cc:1514] Processing DeleteTablet for tablet 88f5fff66a3442af8de842b54a80359e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:07 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:51504
I20260809 08:52:07.045636 10942 tablet_service.cc:1514] Processing DeleteTablet for tablet 88f5fff66a3442af8de842b54a80359e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:07 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:54248
I20260809 08:52:07.045858 11583 tablet_replica.cc:330] T 88f5fff66a3442af8de842b54a80359e P 9ffed0426a0a4f10995bf0e762c9f81f: stopping tablet replica
I20260809 08:52:07.045897 11178 tablet_service.cc:1514] Processing DeleteTablet for tablet 88f5fff66a3442af8de842b54a80359e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:07 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:38236
I20260809 08:52:07.045958 11583 raft_consensus.cc:2229] T 88f5fff66a3442af8de842b54a80359e P 9ffed0426a0a4f10995bf0e762c9f81f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:52:07.045975 10720 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39232:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 08:52:07.046083 11583 raft_consensus.cc:2258] T 88f5fff66a3442af8de842b54a80359e P 9ffed0426a0a4f10995bf0e762c9f81f [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 08:52:07.046109 10720 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:52:07.046324 11585 tablet_replica.cc:330] T 88f5fff66a3442af8de842b54a80359e P b50224ba75a44c32a84878472b542820: stopping tablet replica
I20260809 08:52:07.046452 11585 raft_consensus.cc:2229] T 88f5fff66a3442af8de842b54a80359e P b50224ba75a44c32a84878472b542820 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:52:07.046931 11583 ts_tablet_manager.cc:1905] T 88f5fff66a3442af8de842b54a80359e P 9ffed0426a0a4f10995bf0e762c9f81f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:52:07.047214 11585 raft_consensus.cc:2258] T 88f5fff66a3442af8de842b54a80359e P b50224ba75a44c32a84878472b542820 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:52:07.047295 11409 tablet_service.cc:1467] Processing CreateTablet for tablet c9c3c6bf5a8d4342875b9e3ce3300781 (DEFAULT_TABLE table=test-table [id=35f3258296e54b76bb4eff54b865fd0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:07.047286 10942 tablet_service.cc:1467] Processing CreateTablet for tablet c9c3c6bf5a8d4342875b9e3ce3300781 (DEFAULT_TABLE table=test-table [id=35f3258296e54b76bb4eff54b865fd0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:07.047405 11409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9c3c6bf5a8d4342875b9e3ce3300781. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:07.047405 10942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9c3c6bf5a8d4342875b9e3ce3300781. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:07.047678 11585 ts_tablet_manager.cc:1905] T 88f5fff66a3442af8de842b54a80359e P b50224ba75a44c32a84878472b542820: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:52:07.048786 11546 tablet_bootstrap.cc:492] T c9c3c6bf5a8d4342875b9e3ce3300781 P b50224ba75a44c32a84878472b542820: Bootstrap starting.
I20260809 08:52:07.048871 11540 tablet_bootstrap.cc:492] T c9c3c6bf5a8d4342875b9e3ce3300781 P 9ffed0426a0a4f10995bf0e762c9f81f: Bootstrap starting.
I20260809 08:52:07.048859 11178 tablet_service.cc:1467] Processing CreateTablet for tablet c9c3c6bf5a8d4342875b9e3ce3300781 (DEFAULT_TABLE table=test-table [id=35f3258296e54b76bb4eff54b865fd0b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:07.048964 11178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9c3c6bf5a8d4342875b9e3ce3300781. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:07.049280 11546 tablet_bootstrap.cc:654] T c9c3c6bf5a8d4342875b9e3ce3300781 P b50224ba75a44c32a84878472b542820: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:07.049295 11540 tablet_bootstrap.cc:654] T c9c3c6bf5a8d4342875b9e3ce3300781 P 9ffed0426a0a4f10995bf0e762c9f81f: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:07.050626 11541 tablet_bootstrap.cc:492] T c9c3c6bf5a8d4342875b9e3ce3300781 P 768011a2337d4d49b276f2bae87ac0ca: Bootstrap starting.
I20260809 08:52:07.051122 11541 tablet_bootstrap.cc:654] T c9c3c6bf5a8d4342875b9e3ce3300781 P 768011a2337d4d49b276f2bae87ac0ca: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:07.051674 11585 ts_tablet_manager.cc:1918] T 88f5fff66a3442af8de842b54a80359e P b50224ba75a44c32a84878472b542820: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:52:07.051934 11541 tablet_bootstrap.cc:492] T c9c3c6bf5a8d4342875b9e3ce3300781 P 768011a2337d4d49b276f2bae87ac0ca: No bootstrap required, opened a new log
I20260809 08:52:07.051991 11541 ts_tablet_manager.cc:1397] T c9c3c6bf5a8d4342875b9e3ce3300781 P 768011a2337d4d49b276f2bae87ac0ca: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 08:52:07.052139 11583 ts_tablet_manager.cc:1918] T 88f5fff66a3442af8de842b54a80359e P 9ffed0426a0a4f10995bf0e762c9f81f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:52:07.052227 11583 log.cc:1196] T 88f5fff66a3442af8de842b54a80359e P 9ffed0426a0a4f10995bf0e762c9f81f: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-0/wal/wals/88f5fff66a3442af8de842b54a80359e
I20260809 08:52:07.052194 11541 raft_consensus.cc:348] T c9c3c6bf5a8d4342875b9e3ce3300781 P 768011a2337d4d49b276f2bae87ac0ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 } } peers { permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 } } peers { permanent_uuid: "b50224ba75a44c32a84878472b542820" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45107 } }
I20260809 08:52:07.052263 11541 raft_consensus.cc:374] T c9c3c6bf5a8d4342875b9e3ce3300781 P 768011a2337d4d49b276f2bae87ac0ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:07.052294 11541 raft_consensus.cc:729] T c9c3c6bf5a8d4342875b9e3ce3300781 P 768011a2337d4d49b276f2bae87ac0ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 768011a2337d4d49b276f2bae87ac0ca, State: Initialized, Role: FOLLOWER
I20260809 08:52:07.052335 11541 consensus_queue.cc:260] T c9c3c6bf5a8d4342875b9e3ce3300781 P 768011a2337d4d49b276f2bae87ac0ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 } } peers { permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 } } peers { permanent_uuid: "b50224ba75a44c32a84878472b542820" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45107 } }
I20260809 08:52:07.052419 11541 ts_tablet_manager.cc:1428] T c9c3c6bf5a8d4342875b9e3ce3300781 P 768011a2337d4d49b276f2bae87ac0ca: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 08:52:07.052518 11583 ts_tablet_manager.cc:1939] T 88f5fff66a3442af8de842b54a80359e P 9ffed0426a0a4f10995bf0e762c9f81f: Deleting consensus metadata
I20260809 08:52:07.052646 11585 log.cc:1196] T 88f5fff66a3442af8de842b54a80359e P b50224ba75a44c32a84878472b542820: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-2/wal/wals/88f5fff66a3442af8de842b54a80359e
I20260809 08:52:07.052979 11585 ts_tablet_manager.cc:1939] T 88f5fff66a3442af8de842b54a80359e P b50224ba75a44c32a84878472b542820: Deleting consensus metadata
I20260809 08:52:07.052966 10697 catalog_manager.cc:4841] TS 9ffed0426a0a4f10995bf0e762c9f81f (127.8.154.1:42439): tablet 88f5fff66a3442af8de842b54a80359e (table test-table [id=fe04d1e98fed43d69bef2a5065651c6c]) successfully deleted
I20260809 08:52:07.053308 11586 tablet_replica.cc:330] T 88f5fff66a3442af8de842b54a80359e P 768011a2337d4d49b276f2bae87ac0ca: stopping tablet replica
I20260809 08:52:07.053462 10700 catalog_manager.cc:4841] TS b50224ba75a44c32a84878472b542820 (127.8.154.3:45107): tablet 88f5fff66a3442af8de842b54a80359e (table test-table [id=fe04d1e98fed43d69bef2a5065651c6c]) successfully deleted
I20260809 08:52:07.053635 11586 raft_consensus.cc:2229] T 88f5fff66a3442af8de842b54a80359e P 768011a2337d4d49b276f2bae87ac0ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:52:07.053716 11546 tablet_bootstrap.cc:492] T c9c3c6bf5a8d4342875b9e3ce3300781 P b50224ba75a44c32a84878472b542820: No bootstrap required, opened a new log
I20260809 08:52:07.053771 11546 ts_tablet_manager.cc:1397] T c9c3c6bf5a8d4342875b9e3ce3300781 P b50224ba75a44c32a84878472b542820: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260809 08:52:07.053798 11586 raft_consensus.cc:2258] T 88f5fff66a3442af8de842b54a80359e P 768011a2337d4d49b276f2bae87ac0ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:52:07.053907 11546 raft_consensus.cc:348] T c9c3c6bf5a8d4342875b9e3ce3300781 P b50224ba75a44c32a84878472b542820 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 } } peers { permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 } } peers { permanent_uuid: "b50224ba75a44c32a84878472b542820" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45107 } }
I20260809 08:52:07.053977 11546 raft_consensus.cc:374] T c9c3c6bf5a8d4342875b9e3ce3300781 P b50224ba75a44c32a84878472b542820 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:07.054006 11546 raft_consensus.cc:729] T c9c3c6bf5a8d4342875b9e3ce3300781 P b50224ba75a44c32a84878472b542820 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b50224ba75a44c32a84878472b542820, State: Initialized, Role: FOLLOWER
I20260809 08:52:07.054047 11546 consensus_queue.cc:260] T c9c3c6bf5a8d4342875b9e3ce3300781 P b50224ba75a44c32a84878472b542820 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 } } peers { permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 } } peers { permanent_uuid: "b50224ba75a44c32a84878472b542820" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45107 } }
I20260809 08:52:07.054126 11546 ts_tablet_manager.cc:1428] T c9c3c6bf5a8d4342875b9e3ce3300781 P b50224ba75a44c32a84878472b542820: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:07.054258 11540 tablet_bootstrap.cc:492] T c9c3c6bf5a8d4342875b9e3ce3300781 P 9ffed0426a0a4f10995bf0e762c9f81f: No bootstrap required, opened a new log
I20260809 08:52:07.054304 11540 ts_tablet_manager.cc:1397] T c9c3c6bf5a8d4342875b9e3ce3300781 P 9ffed0426a0a4f10995bf0e762c9f81f: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260809 08:52:07.054445 11540 raft_consensus.cc:348] T c9c3c6bf5a8d4342875b9e3ce3300781 P 9ffed0426a0a4f10995bf0e762c9f81f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 } } peers { permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 } } peers { permanent_uuid: "b50224ba75a44c32a84878472b542820" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45107 } }
I20260809 08:52:07.054535 11540 raft_consensus.cc:374] T c9c3c6bf5a8d4342875b9e3ce3300781 P 9ffed0426a0a4f10995bf0e762c9f81f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:07.054574 11540 raft_consensus.cc:729] T c9c3c6bf5a8d4342875b9e3ce3300781 P 9ffed0426a0a4f10995bf0e762c9f81f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ffed0426a0a4f10995bf0e762c9f81f, State: Initialized, Role: FOLLOWER
I20260809 08:52:07.054617 11540 consensus_queue.cc:260] T c9c3c6bf5a8d4342875b9e3ce3300781 P 9ffed0426a0a4f10995bf0e762c9f81f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 } } peers { permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 } } peers { permanent_uuid: "b50224ba75a44c32a84878472b542820" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45107 } }
I20260809 08:52:07.054711 11540 ts_tablet_manager.cc:1428] T c9c3c6bf5a8d4342875b9e3ce3300781 P 9ffed0426a0a4f10995bf0e762c9f81f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:07.055003 11586 ts_tablet_manager.cc:1905] T 88f5fff66a3442af8de842b54a80359e P 768011a2337d4d49b276f2bae87ac0ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:52:07.056406 11586 ts_tablet_manager.cc:1918] T 88f5fff66a3442af8de842b54a80359e P 768011a2337d4d49b276f2bae87ac0ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:52:07.056466 11586 log.cc:1196] T 88f5fff66a3442af8de842b54a80359e P 768011a2337d4d49b276f2bae87ac0ca: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-1/wal/wals/88f5fff66a3442af8de842b54a80359e
I20260809 08:52:07.056742 11586 ts_tablet_manager.cc:1939] T 88f5fff66a3442af8de842b54a80359e P 768011a2337d4d49b276f2bae87ac0ca: Deleting consensus metadata
I20260809 08:52:07.057137 10695 catalog_manager.cc:4841] TS 768011a2337d4d49b276f2bae87ac0ca (127.8.154.2:38853): tablet 88f5fff66a3442af8de842b54a80359e (table test-table [id=fe04d1e98fed43d69bef2a5065651c6c]) successfully deleted
W20260809 08:52:07.060716 11512 tablet.cc:2369] T c9c3c6bf5a8d4342875b9e3ce3300781 P b50224ba75a44c32a84878472b542820: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:52:07.147625 11555 raft_consensus.cc:482] T c9c3c6bf5a8d4342875b9e3ce3300781 P b50224ba75a44c32a84878472b542820 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:52:07.147749 11555 raft_consensus.cc:504] T c9c3c6bf5a8d4342875b9e3ce3300781 P b50224ba75a44c32a84878472b542820 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 } } peers { permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 } } peers { permanent_uuid: "b50224ba75a44c32a84878472b542820" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45107 } }
I20260809 08:52:07.147941 11555 leader_election.cc:288] T c9c3c6bf5a8d4342875b9e3ce3300781 P b50224ba75a44c32a84878472b542820 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9ffed0426a0a4f10995bf0e762c9f81f (127.8.154.1:42439), 768011a2337d4d49b276f2bae87ac0ca (127.8.154.2:38853)
I20260809 08:52:07.148195 11210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9c3c6bf5a8d4342875b9e3ce3300781" candidate_uuid: "b50224ba75a44c32a84878472b542820" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "768011a2337d4d49b276f2bae87ac0ca" is_pre_election: true
I20260809 08:52:07.148195 10977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9c3c6bf5a8d4342875b9e3ce3300781" candidate_uuid: "b50224ba75a44c32a84878472b542820" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" is_pre_election: true
I20260809 08:52:07.148315 10977 raft_consensus.cc:2454] T c9c3c6bf5a8d4342875b9e3ce3300781 P 9ffed0426a0a4f10995bf0e762c9f81f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b50224ba75a44c32a84878472b542820 in term 0.
I20260809 08:52:07.148319 11210 raft_consensus.cc:2454] T c9c3c6bf5a8d4342875b9e3ce3300781 P 768011a2337d4d49b276f2bae87ac0ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b50224ba75a44c32a84878472b542820 in term 0.
I20260809 08:52:07.148470 11330 leader_election.cc:302] T c9c3c6bf5a8d4342875b9e3ce3300781 P b50224ba75a44c32a84878472b542820 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 768011a2337d4d49b276f2bae87ac0ca, b50224ba75a44c32a84878472b542820; no voters:
I20260809 08:52:07.148610 11555 raft_consensus.cc:2781] T c9c3c6bf5a8d4342875b9e3ce3300781 P b50224ba75a44c32a84878472b542820 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:52:07.148674 11555 raft_consensus.cc:482] T c9c3c6bf5a8d4342875b9e3ce3300781 P b50224ba75a44c32a84878472b542820 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:52:07.148698 11555 raft_consensus.cc:3037] T c9c3c6bf5a8d4342875b9e3ce3300781 P b50224ba75a44c32a84878472b542820 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:07.149680 11555 raft_consensus.cc:504] T c9c3c6bf5a8d4342875b9e3ce3300781 P b50224ba75a44c32a84878472b542820 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 } } peers { permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 } } peers { permanent_uuid: "b50224ba75a44c32a84878472b542820" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45107 } }
I20260809 08:52:07.149824 11555 leader_election.cc:288] T c9c3c6bf5a8d4342875b9e3ce3300781 P b50224ba75a44c32a84878472b542820 [CANDIDATE]: Term 1 election: Requested vote from peers 9ffed0426a0a4f10995bf0e762c9f81f (127.8.154.1:42439), 768011a2337d4d49b276f2bae87ac0ca (127.8.154.2:38853)
I20260809 08:52:07.150039 11210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9c3c6bf5a8d4342875b9e3ce3300781" candidate_uuid: "b50224ba75a44c32a84878472b542820" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "768011a2337d4d49b276f2bae87ac0ca"
I20260809 08:52:07.150067 10977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9c3c6bf5a8d4342875b9e3ce3300781" candidate_uuid: "b50224ba75a44c32a84878472b542820" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ffed0426a0a4f10995bf0e762c9f81f"
I20260809 08:52:07.150128 11210 raft_consensus.cc:3037] T c9c3c6bf5a8d4342875b9e3ce3300781 P 768011a2337d4d49b276f2bae87ac0ca [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:07.150130 10977 raft_consensus.cc:3037] T c9c3c6bf5a8d4342875b9e3ce3300781 P 9ffed0426a0a4f10995bf0e762c9f81f [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:07.151008 11210 raft_consensus.cc:2454] T c9c3c6bf5a8d4342875b9e3ce3300781 P 768011a2337d4d49b276f2bae87ac0ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b50224ba75a44c32a84878472b542820 in term 1.
I20260809 08:52:07.151180 11330 leader_election.cc:302] T c9c3c6bf5a8d4342875b9e3ce3300781 P b50224ba75a44c32a84878472b542820 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 768011a2337d4d49b276f2bae87ac0ca, b50224ba75a44c32a84878472b542820; no voters:
I20260809 08:52:07.151274 11555 raft_consensus.cc:2781] T c9c3c6bf5a8d4342875b9e3ce3300781 P b50224ba75a44c32a84878472b542820 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:52:07.151317 11555 raft_consensus.cc:686] T c9c3c6bf5a8d4342875b9e3ce3300781 P b50224ba75a44c32a84878472b542820 [term 1 LEADER]: Becoming Leader. State: Replica: b50224ba75a44c32a84878472b542820, State: Running, Role: LEADER
I20260809 08:52:07.151365 11555 consensus_queue.cc:237] T c9c3c6bf5a8d4342875b9e3ce3300781 P b50224ba75a44c32a84878472b542820 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 } } peers { permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 } } peers { permanent_uuid: "b50224ba75a44c32a84878472b542820" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45107 } }
I20260809 08:52:07.151556 10977 raft_consensus.cc:2454] T c9c3c6bf5a8d4342875b9e3ce3300781 P 9ffed0426a0a4f10995bf0e762c9f81f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b50224ba75a44c32a84878472b542820 in term 1.
I20260809 08:52:07.152015 10720 catalog_manager.cc:5495] T c9c3c6bf5a8d4342875b9e3ce3300781 P b50224ba75a44c32a84878472b542820 reported cstate change: term changed from 0 to 1, leader changed from <none> to b50224ba75a44c32a84878472b542820 (127.8.154.3). New cstate: current_term: 1 leader_uuid: "b50224ba75a44c32a84878472b542820" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b50224ba75a44c32a84878472b542820" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45107 } health_report { overall_health: HEALTHY } } }
Aug 09 08:52:07 dist-test-slave-t3q3 krb5kdc[10647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265526, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:07.196295 10802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:07.188980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39236 (local address 127.8.154.62:34255)
0809 08:52:07.189055 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:07.189058 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:07.189186 (+ 128us) server_negotiation.cc:407] Connection header received
0809 08:52:07.189284 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:07.189285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:07.189297 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:07.189376 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:07.189868 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:07.190391 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:07.191104 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:07.191302 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:07.191383 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:07.193621 (+ 2238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:07.193622 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:07.193623 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:07.193630 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:07.194886 (+ 1256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:07.195400 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:07.195401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:07.195401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:07.195462 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:07.195667 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:07.195668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:07.195668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:07.195779 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:07.195855 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:07.196019 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 08:52:07.196158 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 08:52:07.196373 11604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:07.188883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:34255 (local address 127.0.0.1:39236)
0809 08:52:07.189098 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:07.189109 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:07.189208 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:07.189382 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:07.189383 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:07.189727 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:07.189778 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:07.189781 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:07.190483 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:07.190484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:07.191015 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:07.191016 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:07.191382 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:07.191382 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:07.191443 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:07.192045 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:07.192049 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:07.193526 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:07.194986 (+ 1460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:07.194994 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:07.194996 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:07.195271 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:07.195531 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:07.195532 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:07.195532 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:07.195595 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:07.195857 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:07.195865 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:07.195911 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 08:52:07.195994 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 08:52:07.196233 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":131,"thread_start_us":100,"threads_started":1}
I20260809 08:52:07.198441 10720 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:39236:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 08:52:07.198704 10720 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:52:07.200376 11178 tablet_service.cc:1467] Processing CreateTablet for tablet 7d13b85040b84efab003f839b3fecb79 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3ff251699f874220a68843805063b6f8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 08:52:07.200431 11409 tablet_service.cc:1467] Processing CreateTablet for tablet 7d13b85040b84efab003f839b3fecb79 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3ff251699f874220a68843805063b6f8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 08:52:07.200421 10942 tablet_service.cc:1467] Processing CreateTablet for tablet 7d13b85040b84efab003f839b3fecb79 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3ff251699f874220a68843805063b6f8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 08:52:07.200613 11409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d13b85040b84efab003f839b3fecb79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:07.200613 10942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d13b85040b84efab003f839b3fecb79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:07.200637 11178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d13b85040b84efab003f839b3fecb79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:07.201973 11541 tablet_bootstrap.cc:492] T 7d13b85040b84efab003f839b3fecb79 P 768011a2337d4d49b276f2bae87ac0ca: Bootstrap starting.
I20260809 08:52:07.202153 11546 tablet_bootstrap.cc:492] T 7d13b85040b84efab003f839b3fecb79 P b50224ba75a44c32a84878472b542820: Bootstrap starting.
I20260809 08:52:07.202462 11546 tablet_bootstrap.cc:654] T 7d13b85040b84efab003f839b3fecb79 P b50224ba75a44c32a84878472b542820: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:07.202481 11541 tablet_bootstrap.cc:654] T 7d13b85040b84efab003f839b3fecb79 P 768011a2337d4d49b276f2bae87ac0ca: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:07.202836 11540 tablet_bootstrap.cc:492] T 7d13b85040b84efab003f839b3fecb79 P 9ffed0426a0a4f10995bf0e762c9f81f: Bootstrap starting.
I20260809 08:52:07.203112 11546 tablet_bootstrap.cc:492] T 7d13b85040b84efab003f839b3fecb79 P b50224ba75a44c32a84878472b542820: No bootstrap required, opened a new log
I20260809 08:52:07.203157 11546 ts_tablet_manager.cc:1397] T 7d13b85040b84efab003f839b3fecb79 P b50224ba75a44c32a84878472b542820: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:07.203181 11541 tablet_bootstrap.cc:492] T 7d13b85040b84efab003f839b3fecb79 P 768011a2337d4d49b276f2bae87ac0ca: No bootstrap required, opened a new log
I20260809 08:52:07.203222 11541 ts_tablet_manager.cc:1397] T 7d13b85040b84efab003f839b3fecb79 P 768011a2337d4d49b276f2bae87ac0ca: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 08:52:07.203255 11540 tablet_bootstrap.cc:654] T 7d13b85040b84efab003f839b3fecb79 P 9ffed0426a0a4f10995bf0e762c9f81f: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:07.203271 11546 raft_consensus.cc:348] T 7d13b85040b84efab003f839b3fecb79 P b50224ba75a44c32a84878472b542820 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 } } peers { permanent_uuid: "b50224ba75a44c32a84878472b542820" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45107 } } peers { permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 } }
I20260809 08:52:07.203343 11546 raft_consensus.cc:374] T 7d13b85040b84efab003f839b3fecb79 P b50224ba75a44c32a84878472b542820 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:07.203364 11546 raft_consensus.cc:729] T 7d13b85040b84efab003f839b3fecb79 P b50224ba75a44c32a84878472b542820 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b50224ba75a44c32a84878472b542820, State: Initialized, Role: FOLLOWER
I20260809 08:52:07.203358 11541 raft_consensus.cc:348] T 7d13b85040b84efab003f839b3fecb79 P 768011a2337d4d49b276f2bae87ac0ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 } } peers { permanent_uuid: "b50224ba75a44c32a84878472b542820" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45107 } } peers { permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 } }
I20260809 08:52:07.203423 11541 raft_consensus.cc:374] T 7d13b85040b84efab003f839b3fecb79 P 768011a2337d4d49b276f2bae87ac0ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:07.203445 11541 raft_consensus.cc:729] T 7d13b85040b84efab003f839b3fecb79 P 768011a2337d4d49b276f2bae87ac0ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 768011a2337d4d49b276f2bae87ac0ca, State: Initialized, Role: FOLLOWER
I20260809 08:52:07.203415 11546 consensus_queue.cc:260] T 7d13b85040b84efab003f839b3fecb79 P b50224ba75a44c32a84878472b542820 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 } } peers { permanent_uuid: "b50224ba75a44c32a84878472b542820" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45107 } } peers { permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 } }
I20260809 08:52:07.203490 11541 consensus_queue.cc:260] T 7d13b85040b84efab003f839b3fecb79 P 768011a2337d4d49b276f2bae87ac0ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 } } peers { permanent_uuid: "b50224ba75a44c32a84878472b542820" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45107 } } peers { permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 } }
I20260809 08:52:07.203523 11546 ts_tablet_manager.cc:1428] T 7d13b85040b84efab003f839b3fecb79 P b50224ba75a44c32a84878472b542820: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 08:52:07.203577 11541 ts_tablet_manager.cc:1428] T 7d13b85040b84efab003f839b3fecb79 P 768011a2337d4d49b276f2bae87ac0ca: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:07.203621 11572 tablet_replica.cc:439] T 7d13b85040b84efab003f839b3fecb79 P b50224ba75a44c32a84878472b542820: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 } } peers { permanent_uuid: "b50224ba75a44c32a84878472b542820" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45107 } } peers { permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 } } }
I20260809 08:52:07.203708 11572 tablet_replica.cc:442] T 7d13b85040b84efab003f839b3fecb79 P b50224ba75a44c32a84878472b542820: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:52:07.203842 11548 tablet_replica.cc:439] T 7d13b85040b84efab003f839b3fecb79 P 768011a2337d4d49b276f2bae87ac0ca: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 } } peers { permanent_uuid: "b50224ba75a44c32a84878472b542820" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45107 } } peers { permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 } } }
I20260809 08:52:07.203953 11548 tablet_replica.cc:442] T 7d13b85040b84efab003f839b3fecb79 P 768011a2337d4d49b276f2bae87ac0ca: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:52:07.204103 11540 tablet_bootstrap.cc:492] T 7d13b85040b84efab003f839b3fecb79 P 9ffed0426a0a4f10995bf0e762c9f81f: No bootstrap required, opened a new log
I20260809 08:52:07.204147 11540 ts_tablet_manager.cc:1397] T 7d13b85040b84efab003f839b3fecb79 P 9ffed0426a0a4f10995bf0e762c9f81f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:07.204293 11540 raft_consensus.cc:348] T 7d13b85040b84efab003f839b3fecb79 P 9ffed0426a0a4f10995bf0e762c9f81f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 } } peers { permanent_uuid: "b50224ba75a44c32a84878472b542820" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45107 } } peers { permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 } }
I20260809 08:52:07.204367 11540 raft_consensus.cc:374] T 7d13b85040b84efab003f839b3fecb79 P 9ffed0426a0a4f10995bf0e762c9f81f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:07.204392 11540 raft_consensus.cc:729] T 7d13b85040b84efab003f839b3fecb79 P 9ffed0426a0a4f10995bf0e762c9f81f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ffed0426a0a4f10995bf0e762c9f81f, State: Initialized, Role: FOLLOWER
I20260809 08:52:07.204438 11540 consensus_queue.cc:260] T 7d13b85040b84efab003f839b3fecb79 P 9ffed0426a0a4f10995bf0e762c9f81f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 } } peers { permanent_uuid: "b50224ba75a44c32a84878472b542820" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45107 } } peers { permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 } }
I20260809 08:52:07.204533 11540 ts_tablet_manager.cc:1428] T 7d13b85040b84efab003f839b3fecb79 P 9ffed0426a0a4f10995bf0e762c9f81f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:07.204596 11552 tablet_replica.cc:439] T 7d13b85040b84efab003f839b3fecb79 P 9ffed0426a0a4f10995bf0e762c9f81f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 } } peers { permanent_uuid: "b50224ba75a44c32a84878472b542820" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45107 } } peers { permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 } } }
I20260809 08:52:07.204715 11552 tablet_replica.cc:442] T 7d13b85040b84efab003f839b3fecb79 P 9ffed0426a0a4f10995bf0e762c9f81f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:52:07.344874 11572 raft_consensus.cc:482] T 7d13b85040b84efab003f839b3fecb79 P b50224ba75a44c32a84878472b542820 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:52:07.345022 11572 raft_consensus.cc:504] T 7d13b85040b84efab003f839b3fecb79 P b50224ba75a44c32a84878472b542820 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 } } peers { permanent_uuid: "b50224ba75a44c32a84878472b542820" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45107 } } peers { permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 } }
I20260809 08:52:07.345240 11572 leader_election.cc:288] T 7d13b85040b84efab003f839b3fecb79 P b50224ba75a44c32a84878472b542820 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9ffed0426a0a4f10995bf0e762c9f81f (127.8.154.1:42439), 768011a2337d4d49b276f2bae87ac0ca (127.8.154.2:38853)
I20260809 08:52:07.345508 10977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d13b85040b84efab003f839b3fecb79" candidate_uuid: "b50224ba75a44c32a84878472b542820" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" is_pre_election: true
I20260809 08:52:07.345508 11210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d13b85040b84efab003f839b3fecb79" candidate_uuid: "b50224ba75a44c32a84878472b542820" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "768011a2337d4d49b276f2bae87ac0ca" is_pre_election: true
I20260809 08:52:07.345618 11210 raft_consensus.cc:2454] T 7d13b85040b84efab003f839b3fecb79 P 768011a2337d4d49b276f2bae87ac0ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b50224ba75a44c32a84878472b542820 in term 0.
I20260809 08:52:07.345618 10977 raft_consensus.cc:2454] T 7d13b85040b84efab003f839b3fecb79 P 9ffed0426a0a4f10995bf0e762c9f81f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b50224ba75a44c32a84878472b542820 in term 0.
I20260809 08:52:07.345782 11330 leader_election.cc:302] T 7d13b85040b84efab003f839b3fecb79 P b50224ba75a44c32a84878472b542820 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 768011a2337d4d49b276f2bae87ac0ca, b50224ba75a44c32a84878472b542820; no voters:
I20260809 08:52:07.345927 11572 raft_consensus.cc:2781] T 7d13b85040b84efab003f839b3fecb79 P b50224ba75a44c32a84878472b542820 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:52:07.345980 11572 raft_consensus.cc:482] T 7d13b85040b84efab003f839b3fecb79 P b50224ba75a44c32a84878472b542820 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:52:07.345995 11572 raft_consensus.cc:3037] T 7d13b85040b84efab003f839b3fecb79 P b50224ba75a44c32a84878472b542820 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:07.346743 11572 raft_consensus.cc:504] T 7d13b85040b84efab003f839b3fecb79 P b50224ba75a44c32a84878472b542820 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 } } peers { permanent_uuid: "b50224ba75a44c32a84878472b542820" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45107 } } peers { permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 } }
I20260809 08:52:07.346890 11572 leader_election.cc:288] T 7d13b85040b84efab003f839b3fecb79 P b50224ba75a44c32a84878472b542820 [CANDIDATE]: Term 1 election: Requested vote from peers 9ffed0426a0a4f10995bf0e762c9f81f (127.8.154.1:42439), 768011a2337d4d49b276f2bae87ac0ca (127.8.154.2:38853)
I20260809 08:52:07.347054 10977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d13b85040b84efab003f839b3fecb79" candidate_uuid: "b50224ba75a44c32a84878472b542820" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ffed0426a0a4f10995bf0e762c9f81f"
I20260809 08:52:07.347083 11210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d13b85040b84efab003f839b3fecb79" candidate_uuid: "b50224ba75a44c32a84878472b542820" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "768011a2337d4d49b276f2bae87ac0ca"
I20260809 08:52:07.347165 10977 raft_consensus.cc:3037] T 7d13b85040b84efab003f839b3fecb79 P 9ffed0426a0a4f10995bf0e762c9f81f [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:07.347175 11210 raft_consensus.cc:3037] T 7d13b85040b84efab003f839b3fecb79 P 768011a2337d4d49b276f2bae87ac0ca [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:07.347884 11210 raft_consensus.cc:2454] T 7d13b85040b84efab003f839b3fecb79 P 768011a2337d4d49b276f2bae87ac0ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b50224ba75a44c32a84878472b542820 in term 1.
I20260809 08:52:07.348037 10977 raft_consensus.cc:2454] T 7d13b85040b84efab003f839b3fecb79 P 9ffed0426a0a4f10995bf0e762c9f81f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b50224ba75a44c32a84878472b542820 in term 1.
I20260809 08:52:07.348104 11330 leader_election.cc:302] T 7d13b85040b84efab003f839b3fecb79 P b50224ba75a44c32a84878472b542820 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 768011a2337d4d49b276f2bae87ac0ca, b50224ba75a44c32a84878472b542820; no voters:
I20260809 08:52:07.348213 11572 raft_consensus.cc:2781] T 7d13b85040b84efab003f839b3fecb79 P b50224ba75a44c32a84878472b542820 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:52:07.348279 11572 raft_consensus.cc:686] T 7d13b85040b84efab003f839b3fecb79 P b50224ba75a44c32a84878472b542820 [term 1 LEADER]: Becoming Leader. State: Replica: b50224ba75a44c32a84878472b542820, State: Running, Role: LEADER
I20260809 08:52:07.348361 11572 consensus_queue.cc:237] T 7d13b85040b84efab003f839b3fecb79 P b50224ba75a44c32a84878472b542820 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 } } peers { permanent_uuid: "b50224ba75a44c32a84878472b542820" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45107 } } peers { permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 } }
I20260809 08:52:07.348675 11555 tablet_replica.cc:439] T 7d13b85040b84efab003f839b3fecb79 P b50224ba75a44c32a84878472b542820: TxnStatusTablet state changed. Reason: New leader b50224ba75a44c32a84878472b542820. Latest consensus state: current_term: 1 leader_uuid: "b50224ba75a44c32a84878472b542820" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 } } peers { permanent_uuid: "b50224ba75a44c32a84878472b542820" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45107 } } peers { permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 } } }
I20260809 08:52:07.348764 11555 tablet_replica.cc:442] T 7d13b85040b84efab003f839b3fecb79 P b50224ba75a44c32a84878472b542820: This TxnStatusTablet replica's current role is: LEADER
I20260809 08:52:07.348912 11612 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 08:52:07.349053 11612 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 08:52:07.349025 10720 catalog_manager.cc:5495] T 7d13b85040b84efab003f839b3fecb79 P b50224ba75a44c32a84878472b542820 reported cstate change: term changed from 0 to 1, leader changed from <none> to b50224ba75a44c32a84878472b542820 (127.8.154.3). New cstate: current_term: 1 leader_uuid: "b50224ba75a44c32a84878472b542820" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b50224ba75a44c32a84878472b542820" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45107 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 } health_report { overall_health: UNKNOWN } } }
I20260809 08:52:07.388955 11537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:07.386314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51508 (local address 127.8.154.3:45107)
0809 08:52:07.386373 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:07.386375 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:07.386391 (+ 16us) server_negotiation.cc:407] Connection header received
0809 08:52:07.386467 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:07.386468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:07.386479 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:07.386528 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:52:07.386658 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:07.387127 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:07.388275 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:07.388457 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:07.388507 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:07.388710 (+ 203us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:52:07.388748 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:52:07.388792 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:07.388875 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 08:52:07.388894 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 08:52:07.388998 11604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:07.386241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:45107 (local address 127.0.0.1:51508)
0809 08:52:07.386330 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:07.386342 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:07.386387 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:07.386536 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:07.386536 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:07.386538 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 08:52:07.386596 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:07.386598 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:07.387210 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:07.387211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:07.388180 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:07.388183 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:07.388528 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:07.388529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:07.388599 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:07.388617 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 08:52:07.388812 (+ 195us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 08:52:07.388821 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 08:52:07.388871 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 08:52:07.388923 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 08:52:07.390028 11210 raft_consensus.cc:1261] T 7d13b85040b84efab003f839b3fecb79 P 768011a2337d4d49b276f2bae87ac0ca [term 1 FOLLOWER]: Refusing update from remote peer b50224ba75a44c32a84878472b542820: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:52:07.390050 10977 raft_consensus.cc:1261] T 7d13b85040b84efab003f839b3fecb79 P 9ffed0426a0a4f10995bf0e762c9f81f [term 1 FOLLOWER]: Refusing update from remote peer b50224ba75a44c32a84878472b542820: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:52:07.390259 11555 consensus_queue.cc:1035] T 7d13b85040b84efab003f839b3fecb79 P b50224ba75a44c32a84878472b542820 [LEADER]: Connected to new peer: Peer: permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:52:07.390365 11572 consensus_queue.cc:1035] T 7d13b85040b84efab003f839b3fecb79 P b50224ba75a44c32a84878472b542820 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:52:07.390812 11548 tablet_replica.cc:439] T 7d13b85040b84efab003f839b3fecb79 P 768011a2337d4d49b276f2bae87ac0ca: TxnStatusTablet state changed. Reason: New leader b50224ba75a44c32a84878472b542820. Latest consensus state: current_term: 1 leader_uuid: "b50224ba75a44c32a84878472b542820" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 } } peers { permanent_uuid: "b50224ba75a44c32a84878472b542820" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45107 } } peers { permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 } } }
I20260809 08:52:07.390905 11548 tablet_replica.cc:442] T 7d13b85040b84efab003f839b3fecb79 P 768011a2337d4d49b276f2bae87ac0ca: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:52:07.391026 11552 tablet_replica.cc:439] T 7d13b85040b84efab003f839b3fecb79 P 9ffed0426a0a4f10995bf0e762c9f81f: TxnStatusTablet state changed. Reason: New leader b50224ba75a44c32a84878472b542820. Latest consensus state: current_term: 1 leader_uuid: "b50224ba75a44c32a84878472b542820" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 } } peers { permanent_uuid: "b50224ba75a44c32a84878472b542820" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45107 } } peers { permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 } } }
I20260809 08:52:07.391113 11552 tablet_replica.cc:442] T 7d13b85040b84efab003f839b3fecb79 P 9ffed0426a0a4f10995bf0e762c9f81f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:52:07.391531 11555 tablet_replica.cc:439] T 7d13b85040b84efab003f839b3fecb79 P b50224ba75a44c32a84878472b542820: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b50224ba75a44c32a84878472b542820" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 } } peers { permanent_uuid: "b50224ba75a44c32a84878472b542820" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45107 } } peers { permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 } } }
I20260809 08:52:07.391608 11555 tablet_replica.cc:442] T 7d13b85040b84efab003f839b3fecb79 P b50224ba75a44c32a84878472b542820: This TxnStatusTablet replica's current role is: LEADER
I20260809 08:52:07.391659 11562 mvcc.cc:205] Tried to move back new op lower bound from 7316543600187895808 to 7316543600019406848. Current Snapshot: MvccSnapshot[applied={T|T < 7316543600187895808}]
I20260809 08:52:07.391793 11564 mvcc.cc:205] Tried to move back new op lower bound from 7316543600187895808 to 7316543600019406848. Current Snapshot: MvccSnapshot[applied={T|T < 7316543600187895808 or (T in {7316543600187895808})}]
I20260809 08:52:07.392127 11552 tablet_replica.cc:439] T 7d13b85040b84efab003f839b3fecb79 P 9ffed0426a0a4f10995bf0e762c9f81f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b50224ba75a44c32a84878472b542820" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 } } peers { permanent_uuid: "b50224ba75a44c32a84878472b542820" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45107 } } peers { permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 } } }
I20260809 08:52:07.392220 11552 tablet_replica.cc:442] T 7d13b85040b84efab003f839b3fecb79 P 9ffed0426a0a4f10995bf0e762c9f81f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:52:07.392727 11559 tablet_replica.cc:439] T 7d13b85040b84efab003f839b3fecb79 P b50224ba75a44c32a84878472b542820: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b50224ba75a44c32a84878472b542820" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 } } peers { permanent_uuid: "b50224ba75a44c32a84878472b542820" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45107 } } peers { permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 } } }
I20260809 08:52:07.392812 11559 tablet_replica.cc:442] T 7d13b85040b84efab003f839b3fecb79 P b50224ba75a44c32a84878472b542820: This TxnStatusTablet replica's current role is: LEADER
I20260809 08:52:07.393406 11548 tablet_replica.cc:439] T 7d13b85040b84efab003f839b3fecb79 P 768011a2337d4d49b276f2bae87ac0ca: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b50224ba75a44c32a84878472b542820" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 } } peers { permanent_uuid: "b50224ba75a44c32a84878472b542820" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45107 } } peers { permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 } } }
I20260809 08:52:07.393489 11548 tablet_replica.cc:442] T 7d13b85040b84efab003f839b3fecb79 P 768011a2337d4d49b276f2bae87ac0ca: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:52:07.398818 11537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:07.396237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:52319 (local address 127.8.154.3:45107)
0809 08:52:07.396276 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:07.396278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:07.396504 (+ 226us) server_negotiation.cc:407] Connection header received
0809 08:52:07.396585 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:07.396586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:07.396614 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:07.396668 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:52:07.396783 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:07.397246 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:07.398064 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:07.398266 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:07.398352 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:07.398526 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:52:07.398562 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:52:07.398639 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:07.398698 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 08:52:07.398720 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 08:52:07.398829 11620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:07.396187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:45107 (local address 127.8.154.3:52319)
0809 08:52:07.396427 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:07.396438 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:07.396520 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:07.396671 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:07.396672 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:07.396674 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 08:52:07.396725 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:07.396726 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:07.397343 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:07.397345 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:07.397975 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:07.397977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:07.398359 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:07.398361 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:07.398411 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:07.398426 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 08:52:07.398639 (+ 213us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 08:52:07.398641 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 08:52:07.398695 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 08:52:07.398743 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":7,"thread_start_us":91,"threads_started":1}
I20260809 08:52:07.401415 10977 raft_consensus.cc:1261] T c9c3c6bf5a8d4342875b9e3ce3300781 P 9ffed0426a0a4f10995bf0e762c9f81f [term 1 FOLLOWER]: Refusing update from remote peer b50224ba75a44c32a84878472b542820: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:52:07.401415 11210 raft_consensus.cc:1261] T c9c3c6bf5a8d4342875b9e3ce3300781 P 768011a2337d4d49b276f2bae87ac0ca [term 1 FOLLOWER]: Refusing update from remote peer b50224ba75a44c32a84878472b542820: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:52:07.401635 11555 consensus_queue.cc:1035] T c9c3c6bf5a8d4342875b9e3ce3300781 P b50224ba75a44c32a84878472b542820 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ffed0426a0a4f10995bf0e762c9f81f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 42439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:52:07.401739 11572 consensus_queue.cc:1035] T c9c3c6bf5a8d4342875b9e3ce3300781 P b50224ba75a44c32a84878472b542820 [LEADER]: Connected to new peer: Peer: permanent_uuid: "768011a2337d4d49b276f2bae87ac0ca" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:52:07.414342 10717 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:52:07.414458 10717 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39232:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:52:07.415110 10717 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9cd9f74df0624cb6a0cacc4027a896fc: Sending DeleteTablet for 3 replicas of tablet c9c3c6bf5a8d4342875b9e3ce3300781
I20260809 08:52:07.415364 10942 tablet_service.cc:1514] Processing DeleteTablet for tablet c9c3c6bf5a8d4342875b9e3ce3300781 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:07 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:54248
I20260809 08:52:07.415463 11583 tablet_replica.cc:330] T c9c3c6bf5a8d4342875b9e3ce3300781 P 9ffed0426a0a4f10995bf0e762c9f81f: stopping tablet replica
I20260809 08:52:07.415510 11583 raft_consensus.cc:2229] T c9c3c6bf5a8d4342875b9e3ce3300781 P 9ffed0426a0a4f10995bf0e762c9f81f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:52:07.415491 11178 tablet_service.cc:1514] Processing DeleteTablet for tablet c9c3c6bf5a8d4342875b9e3ce3300781 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:07 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:38236
I20260809 08:52:07.415545 11583 raft_consensus.cc:2258] T c9c3c6bf5a8d4342875b9e3ce3300781 P 9ffed0426a0a4f10995bf0e762c9f81f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:52:07.415619 11586 tablet_replica.cc:330] T c9c3c6bf5a8d4342875b9e3ce3300781 P 768011a2337d4d49b276f2bae87ac0ca: stopping tablet replica
I20260809 08:52:07.415673 11586 raft_consensus.cc:2229] T c9c3c6bf5a8d4342875b9e3ce3300781 P 768011a2337d4d49b276f2bae87ac0ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:52:07.415704 11586 raft_consensus.cc:2258] T c9c3c6bf5a8d4342875b9e3ce3300781 P 768011a2337d4d49b276f2bae87ac0ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:52:07.416148 11409 tablet_service.cc:1514] Processing DeleteTablet for tablet c9c3c6bf5a8d4342875b9e3ce3300781 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:07 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:51504
I20260809 08:52:07.416280 11585 tablet_replica.cc:330] T c9c3c6bf5a8d4342875b9e3ce3300781 P b50224ba75a44c32a84878472b542820: stopping tablet replica
I20260809 08:52:07.416347 11585 raft_consensus.cc:2229] T c9c3c6bf5a8d4342875b9e3ce3300781 P b50224ba75a44c32a84878472b542820 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:52:07.416404 11585 raft_consensus.cc:2258] T c9c3c6bf5a8d4342875b9e3ce3300781 P b50224ba75a44c32a84878472b542820 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:52:07.416442 11583 ts_tablet_manager.cc:1905] T c9c3c6bf5a8d4342875b9e3ce3300781 P 9ffed0426a0a4f10995bf0e762c9f81f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:52:07.416693 11585 ts_tablet_manager.cc:1905] T c9c3c6bf5a8d4342875b9e3ce3300781 P b50224ba75a44c32a84878472b542820: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:52:07.418278 11583 ts_tablet_manager.cc:1918] T c9c3c6bf5a8d4342875b9e3ce3300781 P 9ffed0426a0a4f10995bf0e762c9f81f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:52:07.418339 11583 log.cc:1196] T c9c3c6bf5a8d4342875b9e3ce3300781 P 9ffed0426a0a4f10995bf0e762c9f81f: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-0/wal/wals/c9c3c6bf5a8d4342875b9e3ce3300781
I20260809 08:52:07.418394 11585 ts_tablet_manager.cc:1918] T c9c3c6bf5a8d4342875b9e3ce3300781 P b50224ba75a44c32a84878472b542820: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:52:07.418447 11585 log.cc:1196] T c9c3c6bf5a8d4342875b9e3ce3300781 P b50224ba75a44c32a84878472b542820: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-2/wal/wals/c9c3c6bf5a8d4342875b9e3ce3300781
I20260809 08:52:07.418565 11583 ts_tablet_manager.cc:1939] T c9c3c6bf5a8d4342875b9e3ce3300781 P 9ffed0426a0a4f10995bf0e762c9f81f: Deleting consensus metadata
I20260809 08:52:07.418658 11585 ts_tablet_manager.cc:1939] T c9c3c6bf5a8d4342875b9e3ce3300781 P b50224ba75a44c32a84878472b542820: Deleting consensus metadata
I20260809 08:52:07.418928 10697 catalog_manager.cc:4841] TS 9ffed0426a0a4f10995bf0e762c9f81f (127.8.154.1:42439): tablet c9c3c6bf5a8d4342875b9e3ce3300781 (table test-table [id=35f3258296e54b76bb4eff54b865fd0b]) successfully deleted
I20260809 08:52:07.418982 10700 catalog_manager.cc:4841] TS b50224ba75a44c32a84878472b542820 (127.8.154.3:45107): tablet c9c3c6bf5a8d4342875b9e3ce3300781 (table test-table [id=35f3258296e54b76bb4eff54b865fd0b]) successfully deleted
I20260809 08:52:07.419283 11586 ts_tablet_manager.cc:1905] T c9c3c6bf5a8d4342875b9e3ce3300781 P 768011a2337d4d49b276f2bae87ac0ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:52:07.420914 11586 ts_tablet_manager.cc:1918] T c9c3c6bf5a8d4342875b9e3ce3300781 P 768011a2337d4d49b276f2bae87ac0ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:52:07.420992 11586 log.cc:1196] T c9c3c6bf5a8d4342875b9e3ce3300781 P 768011a2337d4d49b276f2bae87ac0ca: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265519942966-8808-0/minicluster-data/ts-1/wal/wals/c9c3c6bf5a8d4342875b9e3ce3300781
I20260809 08:52:07.421221 11586 ts_tablet_manager.cc:1939] T c9c3c6bf5a8d4342875b9e3ce3300781 P 768011a2337d4d49b276f2bae87ac0ca: Deleting consensus metadata
I20260809 08:52:07.421541 10695 catalog_manager.cc:4841] TS 768011a2337d4d49b276f2bae87ac0ca (127.8.154.2:38853): tablet c9c3c6bf5a8d4342875b9e3ce3300781 (table test-table [id=35f3258296e54b76bb4eff54b865fd0b]) successfully deleted
Aug 09 08:52:07 dist-test-slave-t3q3 krb5kdc[10647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265526, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.154.1@KRBTEST.COM
I20260809 08:52:07.428797 11536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:07.416395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54256 (local address 127.8.154.1:42439)
0809 08:52:07.416434 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:07.416436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:07.419366 (+ 2930us) server_negotiation.cc:407] Connection header received
0809 08:52:07.419443 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:07.419444 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:07.419456 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:07.419522 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:07.420233 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:07.420964 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:07.422022 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:07.422211 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:07.422280 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:07.425819 (+ 3539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:07.425830 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:07.425831 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:07.425839 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:07.427524 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:07.427894 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:07.427899 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:07.427899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:07.427965 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:07.428206 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:07.428207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:07.428207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:07.428360 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:07.428520 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:07.428594 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 08:52:07.428673 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 08:52:07.429154 11033 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54256
I20260809 08:52:07.437623 10802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:07.430233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39244 (local address 127.8.154.62:34255)
0809 08:52:07.430281 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:07.430282 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:07.430382 (+ 100us) server_negotiation.cc:407] Connection header received
0809 08:52:07.430462 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:07.430463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:07.430471 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:07.430519 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:07.431271 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:07.431755 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:07.432870 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:07.433066 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:07.433115 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:07.434624 (+ 1509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:07.434625 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:07.434625 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:07.434632 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:07.436368 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:07.436801 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:07.436802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:07.436802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:07.436855 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:07.437089 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:07.437090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:07.437090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:07.437244 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:07.437318 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:07.437499 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 08:52:07.437561 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260809 08:52:07.437914 10717 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39244
I20260809 08:52:07.438645 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 10836
I20260809 08:52:07.464053 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 11054
I20260809 08:52:07.475723 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 11301
I20260809 08:52:07.488312 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 10668
2026-08-09T08:52:07Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2771 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:52:07 dist-test-slave-t3q3 krb5kdc[11636](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:52:07 dist-test-slave-t3q3 krb5kdc[11636](info): set up 2 sockets
Aug 09 08:52:07 dist-test-slave-t3q3 krb5kdc[11636](info): commencing operation
krb5kdc: starting...
W20260809 08:52:08.750355 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.235s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:52:08 dist-test-slave-t3q3 krb5kdc[11636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:52:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:52:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
I20260809 08:52:08.870325 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:41637
--webserver_interface=127.8.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:44695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:41637
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:08.953960 11654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:08.954164 11654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:08.954185 11654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:08.955384 11654 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:52:08.955426 11654 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 08:52:08.955440 11654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:08.955453 11654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:08.955505 11654 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:52:08.955523 11654 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:52:08.957000 11654 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:44695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:41637
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:41637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:08.957265 11654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:08.957507 11654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:08.962558 11665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:08.962584 11662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:08.962590 11661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:08.962845 11654 server_base.cc:1034] running on GCE node
I20260809 08:52:08.963124 11654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:08.963400 11654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:08.964553 11654 hybrid_clock.cc:648] HybridClock initialized: now 1786265528964542 us; error 29 us; skew 500 ppm
Aug 09 08:52:08 dist-test-slave-t3q3 krb5kdc[11636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265528, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:08.967075 11654 init.cc:377] Logged in from keytab as kudu/127.8.154.62@KRBTEST.COM (short username kudu)
I20260809 08:52:08.967451 11654 webserver.cc:460] Webserver started at http://127.8.154.62:44159/ using document root <none> and password file <none>
I20260809 08:52:08.967648 11654 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:08.967691 11654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:08.967789 11654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:08.968837 11654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "394558cf3144481bbebd8079b686db41"
format_stamp: "Formatted at 2026-08-09 08:52:08 on dist-test-slave-t3q3"
server_key: "6bde3a0d826c341f2a479aad8fbddb35"
server_key_iv: "2738ec401f1811017849adbc339d9c26"
server_key_version: "encryptionkey@0"
I20260809 08:52:08.969139 11654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "394558cf3144481bbebd8079b686db41"
format_stamp: "Formatted at 2026-08-09 08:52:08 on dist-test-slave-t3q3"
server_key: "6bde3a0d826c341f2a479aad8fbddb35"
server_key_iv: "2738ec401f1811017849adbc339d9c26"
server_key_version: "encryptionkey@0"
I20260809 08:52:08.970458 11654 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 08:52:08.971191 11675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:08.971387 11654 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:52:08.971474 11654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "394558cf3144481bbebd8079b686db41"
format_stamp: "Formatted at 2026-08-09 08:52:08 on dist-test-slave-t3q3"
server_key: "6bde3a0d826c341f2a479aad8fbddb35"
server_key_iv: "2738ec401f1811017849adbc339d9c26"
server_key_version: "encryptionkey@0"
I20260809 08:52:08.971534 11654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:09.007169 11654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:09.009569 11654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:09.009690 11654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:09.014355 11654 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:41637
I20260809 08:52:09.014489 11773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:41637 every 8 connection(s)
I20260809 08:52:09.014803 11654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:52:09.015542 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 11654
I20260809 08:52:09.015553 11774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:09.015657 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:52:09.015844 8808 external_mini_cluster.cc:1428] Setting key 41f41027a8461e35006db087a597f11f
I20260809 08:52:09.018571 11774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 394558cf3144481bbebd8079b686db41: Bootstrap starting.
I20260809 08:52:09.019196 11774 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 394558cf3144481bbebd8079b686db41: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:09.019491 11774 log.cc:824] T 00000000000000000000000000000000 P 394558cf3144481bbebd8079b686db41: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:09.020272 11774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 394558cf3144481bbebd8079b686db41: No bootstrap required, opened a new log
I20260809 08:52:09.022231 11774 raft_consensus.cc:348] T 00000000000000000000000000000000 P 394558cf3144481bbebd8079b686db41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "394558cf3144481bbebd8079b686db41" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 41637 } }
I20260809 08:52:09.022415 11774 raft_consensus.cc:374] T 00000000000000000000000000000000 P 394558cf3144481bbebd8079b686db41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:09.022452 11774 raft_consensus.cc:729] T 00000000000000000000000000000000 P 394558cf3144481bbebd8079b686db41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 394558cf3144481bbebd8079b686db41, State: Initialized, Role: FOLLOWER
Aug 09 08:52:09 dist-test-slave-t3q3 krb5kdc[11636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265528, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:09.022594 11774 consensus_queue.cc:260] T 00000000000000000000000000000000 P 394558cf3144481bbebd8079b686db41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "394558cf3144481bbebd8079b686db41" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 41637 } }
I20260809 08:52:09.022675 11774 raft_consensus.cc:388] T 00000000000000000000000000000000 P 394558cf3144481bbebd8079b686db41 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:52:09.022722 11774 raft_consensus.cc:482] T 00000000000000000000000000000000 P 394558cf3144481bbebd8079b686db41 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:52:09.022773 11774 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 394558cf3144481bbebd8079b686db41 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:09.023484 11774 raft_consensus.cc:504] T 00000000000000000000000000000000 P 394558cf3144481bbebd8079b686db41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "394558cf3144481bbebd8079b686db41" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 41637 } }
I20260809 08:52:09.023612 11774 leader_election.cc:302] T 00000000000000000000000000000000 P 394558cf3144481bbebd8079b686db41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 394558cf3144481bbebd8079b686db41; no voters:
I20260809 08:52:09.023797 11774 leader_election.cc:288] T 00000000000000000000000000000000 P 394558cf3144481bbebd8079b686db41 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:52:09.023828 11781 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 394558cf3144481bbebd8079b686db41 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:52:09.023938 11781 raft_consensus.cc:686] T 00000000000000000000000000000000 P 394558cf3144481bbebd8079b686db41 [term 1 LEADER]: Becoming Leader. State: Replica: 394558cf3144481bbebd8079b686db41, State: Running, Role: LEADER
I20260809 08:52:09.024132 11774 sys_catalog.cc:564] T 00000000000000000000000000000000 P 394558cf3144481bbebd8079b686db41 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:52:09.024132 11781 consensus_queue.cc:237] T 00000000000000000000000000000000 P 394558cf3144481bbebd8079b686db41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "394558cf3144481bbebd8079b686db41" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 41637 } }
I20260809 08:52:09.024796 11783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 394558cf3144481bbebd8079b686db41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "394558cf3144481bbebd8079b686db41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "394558cf3144481bbebd8079b686db41" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 41637 } } }
I20260809 08:52:09.024891 11783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 394558cf3144481bbebd8079b686db41 [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:09.025363 11785 sys_catalog.cc:455] T 00000000000000000000000000000000 P 394558cf3144481bbebd8079b686db41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 394558cf3144481bbebd8079b686db41. Latest consensus state: current_term: 1 leader_uuid: "394558cf3144481bbebd8079b686db41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "394558cf3144481bbebd8079b686db41" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 41637 } } }
I20260809 08:52:09.025449 11785 sys_catalog.cc:458] T 00000000000000000000000000000000 P 394558cf3144481bbebd8079b686db41 [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:09.025676 11796 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:52:09.026209 11796 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:52:09.027331 11778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:09.016450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49874 (local address 127.8.154.62:41637)
0809 08:52:09.016658 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:09.016666 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:09.016692 (+ 26us) server_negotiation.cc:407] Connection header received
0809 08:52:09.017131 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:09.017138 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:09.017186 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:09.017345 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:09.018005 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.019006 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.019899 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.020423 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.020568 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:09.023217 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:09.023221 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:09.023222 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:09.023231 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:09.025462 (+ 2231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:09.025882 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:09.025884 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:09.025885 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:09.026011 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:09.026258 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:09.026259 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:09.026259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:09.026576 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:09.026722 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:09.026973 (+ 251us) server_negotiation.cc:299] Negotiation successful
0809 08:52:09.027122 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":82,"threads_started":1}
I20260809 08:52:09.027815 11796 catalog_manager.cc:1347] Generated new cluster ID: 066106015b3d4fbb971a28923426e14b
I20260809 08:52:09.027869 11796 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:52:09.043648 11796 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:52:09.044281 11796 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:52:09.049968 11796 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 394558cf3144481bbebd8079b686db41: Generated new TSK 0
I20260809 08:52:09.050158 11796 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab.
Entry for principal kudu/127.8.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab.
I20260809 08:52:09.086786 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.1:0
--local_ip_for_outbound_sockets=127.8.154.1
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:41637
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:44695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:09.196164 11825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:09.196374 11825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:09.196400 11825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:09.197516 11825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:09.197553 11825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:09.197573 11825 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 08:52:09.197585 11825 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 08:52:09.197602 11825 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.1
I20260809 08:52:09.198992 11825 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:44695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.154.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.154.62:41637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.1
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:09.199180 11825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:09.199357 11825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:09.205276 11835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:09.205349 11825 server_base.cc:1034] running on GCE node
W20260809 08:52:09.205289 11839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:09.205291 11837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:09.205719 11825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:09.205986 11825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:09.207161 11825 hybrid_clock.cc:648] HybridClock initialized: now 1786265529207139 us; error 38 us; skew 500 ppm
Aug 09 08:52:09 dist-test-slave-t3q3 krb5kdc[11636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265529, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:09.209751 11825 init.cc:377] Logged in from keytab as kudu/127.8.154.1@KRBTEST.COM (short username kudu)
I20260809 08:52:09.210242 11825 webserver.cc:460] Webserver started at http://127.8.154.1:36831/ using document root <none> and password file <none>
I20260809 08:52:09.210467 11825 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:09.210536 11825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:09.210685 11825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:09.211743 11825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-0/data/instance:
uuid: "3f0c22abd96e44ed8d7b4616852a011a"
format_stamp: "Formatted at 2026-08-09 08:52:09 on dist-test-slave-t3q3"
server_key: "60da88b4b3833f453179eec9bc0d78a1"
server_key_iv: "dbb29415ad9db2d8819512bb10be6791"
server_key_version: "encryptionkey@0"
I20260809 08:52:09.212244 11825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance:
uuid: "3f0c22abd96e44ed8d7b4616852a011a"
format_stamp: "Formatted at 2026-08-09 08:52:09 on dist-test-slave-t3q3"
server_key: "60da88b4b3833f453179eec9bc0d78a1"
server_key_iv: "dbb29415ad9db2d8819512bb10be6791"
server_key_version: "encryptionkey@0"
I20260809 08:52:09.213677 11825 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:52:09.214437 11853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:09.214614 11825 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:52:09.214715 11825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-0/wal
uuid: "3f0c22abd96e44ed8d7b4616852a011a"
format_stamp: "Formatted at 2026-08-09 08:52:09 on dist-test-slave-t3q3"
server_key: "60da88b4b3833f453179eec9bc0d78a1"
server_key_iv: "dbb29415ad9db2d8819512bb10be6791"
server_key_version: "encryptionkey@0"
I20260809 08:52:09.214804 11825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:09.226480 11825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:09.228613 11825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:09.228788 11825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:09.229372 11825 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:09.229431 11825 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:09.229480 11825 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:09.229502 11825 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 08:52:09 dist-test-slave-t3q3 krb5kdc[11636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265529, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:09.237210 11825 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.1:45991
I20260809 08:52:09.237663 11825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
I20260809 08:52:09.237644 12037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.1:45991 every 8 connection(s)
I20260809 08:52:09.239638 11778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:09.229817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:58461 (local address 127.8.154.62:41637)
0809 08:52:09.229889 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:09.229891 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:09.230141 (+ 250us) server_negotiation.cc:407] Connection header received
0809 08:52:09.230631 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:09.230633 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:09.230648 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:09.230738 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:09.231369 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.232225 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.234001 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.234293 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.234386 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:09.236721 (+ 2335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:09.236723 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:09.236723 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:09.236731 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:09.238448 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:09.238821 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:09.238822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:09.238822 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:09.238877 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:09.239114 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:09.239114 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:09.239114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:09.239219 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:09.239275 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:09.239509 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 08:52:09.239575 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 08:52:09.239792 11869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:09.229803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:41637 (local address 127.8.154.1:58461)
0809 08:52:09.230001 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:09.230042 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:09.230527 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:09.230788 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:09.230791 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:09.231048 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:09.231262 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:09.231268 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.232352 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.232353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:09.233901 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:09.233907 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.234410 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.234412 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:09.234546 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:09.235118 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:09.235127 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:09.236616 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:09.238527 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:09.238531 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:09.238533 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:09.238752 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:09.238941 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:09.238941 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:09.238942 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:09.239055 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:09.239280 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:09.239285 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:09.239397 (+ 112us) client_negotiation.cc:769] Sending connection context
0809 08:52:09.239476 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 08:52:09.239611 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":122,"threads_started":1}
I20260809 08:52:09.240304 12039 heartbeater.cc:347] Connected to a master server at 127.8.154.62:41637
I20260809 08:52:09.240389 12039 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:09.240573 12039 heartbeater.cc:511] Master 127.8.154.62:41637 requested a full tablet report, sending...
I20260809 08:52:09.241129 11698 ts_manager.cc:194] Registered new tserver with Master: 3f0c22abd96e44ed8d7b4616852a011a (127.8.154.1:45991)
I20260809 08:52:09.241832 11698 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.1@KRBTEST.COM'} at 127.8.154.1:58461
I20260809 08:52:09.242888 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 11825
I20260809 08:52:09.243005 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance
I20260809 08:52:09.243243 8808 external_mini_cluster.cc:1428] Setting key 4af0a29e99a9156f1b53c4e39627528b
I20260809 08:52:09.246903 12049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:09.241321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:41637 (local address 127.8.154.1:38239)
0809 08:52:09.241459 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:09.241474 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:09.241550 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:09.241679 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:09.241680 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:09.241856 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:09.241939 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:09.241942 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.242647 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.242648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:09.243253 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:09.243255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.243633 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.243634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:09.243709 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:09.244147 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:09.244152 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:09.244421 (+ 269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:09.245830 (+ 1409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:09.245831 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:09.245832 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:09.246044 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:09.246270 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:09.246271 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:09.246271 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:09.246334 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:09.246625 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:09.246626 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:09.246689 (+ 63us) client_negotiation.cc:769] Sending connection context
0809 08:52:09.246764 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 08:52:09.246842 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":85,"threads_started":1}
I20260809 08:52:09.247035 11778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:09.241362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:38239 (local address 127.8.154.62:41637)
0809 08:52:09.241395 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:09.241397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:09.241534 (+ 137us) server_negotiation.cc:407] Connection header received
0809 08:52:09.241608 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:09.241609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:09.241619 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:09.241674 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:09.242032 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.242547 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.243340 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.243549 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.243622 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:09.244521 (+ 899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:09.244522 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:09.244523 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:09.244529 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:09.245746 (+ 1217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:09.246123 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:09.246125 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:09.246125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:09.246180 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:09.246412 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:09.246413 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:09.246413 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:09.246525 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:09.246609 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:09.246813 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 08:52:09.246902 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
WARNING: no policy specified for kudu/127.8.154.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab.
Entry for principal kudu/127.8.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab.
I20260809 08:52:09.270670 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.2:0
--local_ip_for_outbound_sockets=127.8.154.2
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:41637
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:44695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:09.353582 12056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:09.353796 12056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:09.353821 12056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:09.354971 12056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:09.355016 12056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:09.355037 12056 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 08:52:09.355089 12056 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 08:52:09.355115 12056 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.2
I20260809 08:52:09.356617 12056 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:44695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.154.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.154.62:41637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.2
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:09.356873 12056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:09.357108 12056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:09.362445 12067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:09.362411 12068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:09.362432 12077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:09.362465 12056 server_base.cc:1034] running on GCE node
I20260809 08:52:09.362813 12056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:09.363070 12056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:09.364234 12056 hybrid_clock.cc:648] HybridClock initialized: now 1786265529364220 us; error 48 us; skew 500 ppm
Aug 09 08:52:09 dist-test-slave-t3q3 krb5kdc[11636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265529, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:09.366671 12056 init.cc:377] Logged in from keytab as kudu/127.8.154.2@KRBTEST.COM (short username kudu)
I20260809 08:52:09.367168 12056 webserver.cc:460] Webserver started at http://127.8.154.2:35603/ using document root <none> and password file <none>
I20260809 08:52:09.367403 12056 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:09.367473 12056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:09.367640 12056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:09.368906 12056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-1/data/instance:
uuid: "106032240a5c4c63bc2816062da95e6f"
format_stamp: "Formatted at 2026-08-09 08:52:09 on dist-test-slave-t3q3"
server_key: "b2148b65b9ed2fa7e9fa2636174c8460"
server_key_iv: "0832485bc6d9da488c36730e9376a2b9"
server_key_version: "encryptionkey@0"
I20260809 08:52:09.369396 12056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance:
uuid: "106032240a5c4c63bc2816062da95e6f"
format_stamp: "Formatted at 2026-08-09 08:52:09 on dist-test-slave-t3q3"
server_key: "b2148b65b9ed2fa7e9fa2636174c8460"
server_key_iv: "0832485bc6d9da488c36730e9376a2b9"
server_key_version: "encryptionkey@0"
I20260809 08:52:09.370859 12056 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:52:09.371625 12086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:09.371830 12056 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:09.371939 12056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-1/wal
uuid: "106032240a5c4c63bc2816062da95e6f"
format_stamp: "Formatted at 2026-08-09 08:52:09 on dist-test-slave-t3q3"
server_key: "b2148b65b9ed2fa7e9fa2636174c8460"
server_key_iv: "0832485bc6d9da488c36730e9376a2b9"
server_key_version: "encryptionkey@0"
I20260809 08:52:09.372014 12056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:09.388372 12056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:09.390473 12056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:09.390645 12056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:09.391357 12056 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:09.391428 12056 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:09.391461 12056 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:09.391481 12056 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 08:52:09 dist-test-slave-t3q3 krb5kdc[11636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265529, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:09.400372 12056 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.2:37729
I20260809 08:52:09.400738 12056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
I20260809 08:52:09.401569 11778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:09.391705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:41533 (local address 127.8.154.62:41637)
0809 08:52:09.391757 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:09.391759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:09.391964 (+ 205us) server_negotiation.cc:407] Connection header received
0809 08:52:09.392533 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:09.392535 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:09.392549 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:09.392630 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:09.393250 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.393770 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.394853 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.395090 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.395171 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:09.397854 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:09.397855 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:09.397856 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:09.397863 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:09.399770 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:09.400242 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:09.400243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:09.400244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:09.400314 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:09.400623 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:09.400624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:09.400624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:09.400750 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:09.400835 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:09.401386 (+ 551us) server_negotiation.cc:299] Negotiation successful
0809 08:52:09.401478 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 08:52:09.401226 12270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.2:37729 every 8 connection(s)
I20260809 08:52:09.401751 12100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:09.391691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:41637 (local address 127.8.154.2:41533)
0809 08:52:09.391845 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:09.391883 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:09.392422 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:09.392701 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:09.392705 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:09.392941 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:09.393143 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:09.393147 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.393871 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.393873 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:09.394743 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:09.394746 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.395178 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.395180 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:09.395392 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:09.395951 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:09.395963 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:09.397736 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:09.399858 (+ 2122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:09.399864 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:09.399865 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:09.400170 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:09.400387 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:09.400388 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:09.400388 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:09.400520 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:09.401019 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:09.401051 (+ 32us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:09.401218 (+ 167us) client_negotiation.cc:769] Sending connection context
0809 08:52:09.401342 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 08:52:09.401506 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":92,"threads_started":1}
I20260809 08:52:09.402350 12271 heartbeater.cc:347] Connected to a master server at 127.8.154.62:41637
I20260809 08:52:09.402433 12271 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:09.402630 12271 heartbeater.cc:511] Master 127.8.154.62:41637 requested a full tablet report, sending...
I20260809 08:52:09.403059 11697 ts_manager.cc:194] Registered new tserver with Master: 106032240a5c4c63bc2816062da95e6f (127.8.154.2:37729)
I20260809 08:52:09.403471 11697 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.2@KRBTEST.COM'} at 127.8.154.2:41533
I20260809 08:52:09.406263 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 12056
I20260809 08:52:09.406383 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance
I20260809 08:52:09.406569 8808 external_mini_cluster.cc:1428] Setting key 983ea14f93c7058dc3d00c1c3d66ae4a
I20260809 08:52:09.409489 11778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:09.403596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:48915 (local address 127.8.154.62:41637)
0809 08:52:09.403633 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:09.403635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:09.403804 (+ 169us) server_negotiation.cc:407] Connection header received
0809 08:52:09.403860 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:09.403861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:09.403871 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:09.403930 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:09.404315 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.404767 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.405423 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.405613 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.405664 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:09.406734 (+ 1070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:09.406736 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:09.406737 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:09.406744 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:09.408253 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:09.408623 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:09.408625 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:09.408625 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:09.408689 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:09.408915 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:09.408916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:09.408916 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:09.409071 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:09.409150 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:09.409323 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 08:52:09.409395 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 08:52:09.409503 12280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:09.403557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:41637 (local address 127.8.154.2:48915)
0809 08:52:09.403720 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:09.403741 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:09.403794 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:09.403940 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:09.403941 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:09.404172 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:09.404240 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:09.404242 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.404852 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.404853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:09.405350 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:09.405351 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.405679 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.405680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:09.405743 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:09.406259 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:09.406263 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:09.406604 (+ 341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:09.408353 (+ 1749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:09.408354 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:09.408355 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:09.408533 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:09.408767 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:09.408768 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:09.408768 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:09.408841 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:09.409154 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:09.409155 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:09.409202 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 08:52:09.409286 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 08:52:09.409397 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":91,"threads_started":1}
WARNING: no policy specified for kudu/127.8.154.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab.
Entry for principal kudu/127.8.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab.
I20260809 08:52:09.434589 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.3:0
--local_ip_for_outbound_sockets=127.8.154.3
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:41637
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:44695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:09.518874 12284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:09.519088 12284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:09.519112 12284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:09.520287 12284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:09.520330 12284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:09.520351 12284 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 08:52:09.520363 12284 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 08:52:09.520380 12284 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.3
I20260809 08:52:09.521803 12284 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:44695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.154.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.154.62:41637
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.3
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:09.522004 12284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:09.522189 12284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:09.527365 12292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:09.527400 12295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:09.527654 12284 server_base.cc:1034] running on GCE node
W20260809 08:52:09.527420 12298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:09.527958 12284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:09.528262 12284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:09.529435 12284 hybrid_clock.cc:648] HybridClock initialized: now 1786265529529430 us; error 37 us; skew 500 ppm
Aug 09 08:52:09 dist-test-slave-t3q3 krb5kdc[11636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265529, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:09.532187 12284 init.cc:377] Logged in from keytab as kudu/127.8.154.3@KRBTEST.COM (short username kudu)
I20260809 08:52:09.532687 12284 webserver.cc:460] Webserver started at http://127.8.154.3:42365/ using document root <none> and password file <none>
I20260809 08:52:09.532951 12284 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:09.533030 12284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:09.533185 12284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:09.534271 12284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-2/data/instance:
uuid: "30e8e1019603485185b0556dda0d8d37"
format_stamp: "Formatted at 2026-08-09 08:52:09 on dist-test-slave-t3q3"
server_key: "8bcdb68ce2b8a2d1876acfe1a7c4bf63"
server_key_iv: "b0794e1bf12daa7cdf5bad773c6744a9"
server_key_version: "encryptionkey@0"
I20260809 08:52:09.534652 12284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance:
uuid: "30e8e1019603485185b0556dda0d8d37"
format_stamp: "Formatted at 2026-08-09 08:52:09 on dist-test-slave-t3q3"
server_key: "8bcdb68ce2b8a2d1876acfe1a7c4bf63"
server_key_iv: "b0794e1bf12daa7cdf5bad773c6744a9"
server_key_version: "encryptionkey@0"
I20260809 08:52:09.536278 12284 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:52:09.537124 12307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:09.537313 12284 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:09.537447 12284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-2/wal
uuid: "30e8e1019603485185b0556dda0d8d37"
format_stamp: "Formatted at 2026-08-09 08:52:09 on dist-test-slave-t3q3"
server_key: "8bcdb68ce2b8a2d1876acfe1a7c4bf63"
server_key_iv: "b0794e1bf12daa7cdf5bad773c6744a9"
server_key_version: "encryptionkey@0"
I20260809 08:52:09.537531 12284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:09.555166 12284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:09.557978 12284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:09.558176 12284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:09.558836 12284 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:09.558905 12284 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:09.558954 12284 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:09.558980 12284 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 08:52:09 dist-test-slave-t3q3 krb5kdc[11636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265529, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:09.566895 12284 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.3:43665
I20260809 08:52:09.566972 12517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.3:43665 every 8 connection(s)
I20260809 08:52:09.567368 12284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
I20260809 08:52:09.569531 11778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:09.559400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:50269 (local address 127.8.154.62:41637)
0809 08:52:09.559445 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:09.559447 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:09.559761 (+ 314us) server_negotiation.cc:407] Connection header received
0809 08:52:09.560384 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:09.560385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:09.560398 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:09.560462 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:09.561311 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.562176 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.563844 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.564207 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.564312 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:09.566662 (+ 2350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:09.566663 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:09.566664 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:09.566672 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:09.568311 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:09.568700 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:09.568701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:09.568702 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:09.568770 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:09.568997 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:09.568998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:09.568998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:09.569107 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:09.569160 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:09.569401 (+ 241us) server_negotiation.cc:299] Negotiation successful
0809 08:52:09.569468 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 08:52:09.569658 12321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:09.559413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:41637 (local address 127.8.154.3:50269)
0809 08:52:09.559635 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:09.559672 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:09.560272 (+ 600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:09.560541 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:09.560545 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:09.560841 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:09.561196 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:09.561204 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.562317 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.562318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:09.563746 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:09.563750 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.564313 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.564314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:09.564473 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:09.564996 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:09.565013 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:09.566544 (+ 1531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:09.568379 (+ 1835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:09.568384 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:09.568386 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:09.568608 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:09.568838 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:09.568839 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:09.568839 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:09.568938 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:09.569165 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:09.569170 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:09.569280 (+ 110us) client_negotiation.cc:769] Sending connection context
0809 08:52:09.569365 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 08:52:09.569481 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":130,"threads_started":1}
I20260809 08:52:09.570245 12518 heartbeater.cc:347] Connected to a master server at 127.8.154.62:41637
I20260809 08:52:09.570355 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 12284
I20260809 08:52:09.570391 12518 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:09.570462 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance
I20260809 08:52:09.570655 12518 heartbeater.cc:511] Master 127.8.154.62:41637 requested a full tablet report, sending...
I20260809 08:52:09.570678 8808 external_mini_cluster.cc:1428] Setting key a1e79ca6c89288fbad40e5cb8dee9549
I20260809 08:52:09.571154 11698 ts_manager.cc:194] Registered new tserver with Master: 30e8e1019603485185b0556dda0d8d37 (127.8.154.3:43665)
I20260809 08:52:09.571610 8808 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:52:09.571642 11698 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.3@KRBTEST.COM'} at 127.8.154.3:50269
I20260809 08:52:09.577932 11778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:09.571450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:45621 (local address 127.8.154.62:41637)
0809 08:52:09.571484 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:09.571486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:09.571496 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:52:09.571540 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:09.571541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:09.571550 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:09.571609 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:09.572567 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.573021 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.574117 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.574306 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.574409 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:09.575339 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:09.575340 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:09.575340 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:09.575347 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:09.576780 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:09.577148 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:09.577149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:09.577149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:09.577206 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:09.577420 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:09.577421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:09.577421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:09.577529 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:09.577609 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:09.577774 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 08:52:09.577842 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 08:52:09.577939 12528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:09.571259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:41637 (local address 127.8.154.3:45621)
0809 08:52:09.571423 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:09.571434 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:09.571486 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:09.572149 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:09.572151 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:09.572384 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:09.572465 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:09.572467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.573127 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.573128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:09.574012 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:09.574015 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.574412 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.574414 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:09.574498 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:09.574942 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:09.574947 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:09.575229 (+ 282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:09.576866 (+ 1637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:09.576867 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:09.576868 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:09.577046 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:09.577285 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:09.577286 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:09.577286 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:09.577353 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:09.577604 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:09.577605 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:09.577652 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 08:52:09.577735 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 08:52:09.577842 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":102,"threads_started":1}
Aug 09 08:52:09 dist-test-slave-t3q3 krb5kdc[11636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265529, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 08:52:09 dist-test-slave-t3q3 krb5kdc[11636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265529, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:09.595793 11778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:09.589381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48414 (local address 127.8.154.62:41637)
0809 08:52:09.589442 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:09.589444 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:09.589453 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:52:09.589480 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:09.589481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:09.589497 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:09.589550 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:09.590089 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.590560 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.591315 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.591517 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.591570 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:09.593672 (+ 2102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:09.593674 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:09.593674 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:09.593680 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:09.594831 (+ 1151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:09.595145 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:09.595146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:09.595146 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:09.595200 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:09.595382 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:09.595383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:09.595383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:09.595485 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:09.595574 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:09.595675 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 08:52:09.595732 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 08:52:09.597643 11698 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48414:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:52:09.599282 11698 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:52:09.606426 12558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:09.601791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33100 (local address 127.8.154.3:43665)
0809 08:52:09.602121 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:09.602128 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:09.602139 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:52:09.602179 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:09.602181 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:09.602197 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:09.602273 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:09.602597 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.603783 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.605205 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.605886 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.605953 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:09.606010 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:09.606282 (+ 272us) server_negotiation.cc:299] Negotiation successful
0809 08:52:09.606325 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":76,"threads_started":1}
I20260809 08:52:09.606422 12554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:09.601669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:43665 (local address 127.0.0.1:33100)
0809 08:52:09.601859 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:09.601901 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:09.601969 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:09.602303 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:09.602306 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:09.602310 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:09.602508 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:09.602510 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.603862 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.603863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:09.605114 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:09.605115 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.606033 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.606034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:09.606144 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:09.606163 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 08:52:09.606252 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 08:52:09.606323 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":37,"thread_start_us":78,"threads_started":1}
I20260809 08:52:09.606421 12553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:09.601679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:45991 (local address 127.0.0.1:54252)
0809 08:52:09.601975 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:09.601987 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:09.602042 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:09.602637 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:09.602638 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:09.602643 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:09.602704 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:09.602706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.603950 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.603951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:09.604988 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:09.604991 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.606031 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.606033 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:09.606144 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:09.606163 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 08:52:09.606252 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 08:52:09.606323 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":4,"thread_start_us":88,"threads_started":1}
I20260809 08:52:09.607096 12560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:09.601976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:37729 (local address 127.0.0.1:52658)
0809 08:52:09.602101 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:09.602112 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:09.602157 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:09.602637 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:09.602638 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:09.602641 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:09.602716 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:09.602717 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.603684 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.603685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:09.605112 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:09.605115 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.606903 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.606904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:09.606961 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:09.606973 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 08:52:09.607016 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 08:52:09.607047 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":83,"thread_start_us":83,"threads_started":1}
I20260809 08:52:09.607216 12561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:09.602193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52658 (local address 127.8.154.2:37729)
0809 08:52:09.602463 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:09.602466 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:09.602476 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:52:09.602522 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:09.602524 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:09.602539 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:09.602793 (+ 254us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:09.602849 (+ 56us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.603604 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.606049 (+ 2445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.606811 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.606894 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:09.606955 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:09.607044 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 08:52:09.607093 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":115,"threads_started":1}
I20260809 08:52:09.607776 12391 tablet_service.cc:1467] Processing CreateTablet for tablet 641ac19be124433caea3225631bafd67 (DEFAULT_TABLE table=test-table [id=130e86c8a0b94ffd8091bfb037f020cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:09.607777 12162 tablet_service.cc:1467] Processing CreateTablet for tablet 641ac19be124433caea3225631bafd67 (DEFAULT_TABLE table=test-table [id=130e86c8a0b94ffd8091bfb037f020cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:09.608212 12391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 641ac19be124433caea3225631bafd67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:09.608212 12162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 641ac19be124433caea3225631bafd67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:09.608330 12559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:09.601863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54252 (local address 127.8.154.1:45991)
0809 08:52:09.602411 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:09.602415 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:09.602425 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:52:09.602466 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:09.602468 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:09.602485 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:09.602567 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:09.602800 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.603887 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.605080 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.605851 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.605943 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:09.606011 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:09.608180 (+ 2169us) server_negotiation.cc:299] Negotiation successful
0809 08:52:09.608218 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":86,"threads_started":1}
I20260809 08:52:09.608942 11943 tablet_service.cc:1467] Processing CreateTablet for tablet 641ac19be124433caea3225631bafd67 (DEFAULT_TABLE table=test-table [id=130e86c8a0b94ffd8091bfb037f020cf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:09.609269 11943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 641ac19be124433caea3225631bafd67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:09.611727 12562 tablet_bootstrap.cc:492] T 641ac19be124433caea3225631bafd67 P 30e8e1019603485185b0556dda0d8d37: Bootstrap starting.
I20260809 08:52:09.612010 12565 tablet_bootstrap.cc:492] T 641ac19be124433caea3225631bafd67 P 106032240a5c4c63bc2816062da95e6f: Bootstrap starting.
I20260809 08:52:09.612465 12566 tablet_bootstrap.cc:492] T 641ac19be124433caea3225631bafd67 P 3f0c22abd96e44ed8d7b4616852a011a: Bootstrap starting.
I20260809 08:52:09.612766 12565 tablet_bootstrap.cc:654] T 641ac19be124433caea3225631bafd67 P 106032240a5c4c63bc2816062da95e6f: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:09.612916 12562 tablet_bootstrap.cc:654] T 641ac19be124433caea3225631bafd67 P 30e8e1019603485185b0556dda0d8d37: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:09.613183 12565 log.cc:824] T 641ac19be124433caea3225631bafd67 P 106032240a5c4c63bc2816062da95e6f: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:09.613219 12566 tablet_bootstrap.cc:654] T 641ac19be124433caea3225631bafd67 P 3f0c22abd96e44ed8d7b4616852a011a: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:09.613531 12562 log.cc:824] T 641ac19be124433caea3225631bafd67 P 30e8e1019603485185b0556dda0d8d37: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:09.613550 12566 log.cc:824] T 641ac19be124433caea3225631bafd67 P 3f0c22abd96e44ed8d7b4616852a011a: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:09.614253 12566 tablet_bootstrap.cc:492] T 641ac19be124433caea3225631bafd67 P 3f0c22abd96e44ed8d7b4616852a011a: No bootstrap required, opened a new log
I20260809 08:52:09.614259 12562 tablet_bootstrap.cc:492] T 641ac19be124433caea3225631bafd67 P 30e8e1019603485185b0556dda0d8d37: No bootstrap required, opened a new log
I20260809 08:52:09.614313 12566 ts_tablet_manager.cc:1397] T 641ac19be124433caea3225631bafd67 P 3f0c22abd96e44ed8d7b4616852a011a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:52:09.614341 12562 ts_tablet_manager.cc:1397] T 641ac19be124433caea3225631bafd67 P 30e8e1019603485185b0556dda0d8d37: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 08:52:09.615715 12562 raft_consensus.cc:348] T 641ac19be124433caea3225631bafd67 P 30e8e1019603485185b0556dda0d8d37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } } peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } }
I20260809 08:52:09.615844 12562 raft_consensus.cc:374] T 641ac19be124433caea3225631bafd67 P 30e8e1019603485185b0556dda0d8d37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:09.615720 12566 raft_consensus.cc:348] T 641ac19be124433caea3225631bafd67 P 3f0c22abd96e44ed8d7b4616852a011a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } } peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } }
I20260809 08:52:09.615867 12562 raft_consensus.cc:729] T 641ac19be124433caea3225631bafd67 P 30e8e1019603485185b0556dda0d8d37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30e8e1019603485185b0556dda0d8d37, State: Initialized, Role: FOLLOWER
I20260809 08:52:09.615934 12566 raft_consensus.cc:374] T 641ac19be124433caea3225631bafd67 P 3f0c22abd96e44ed8d7b4616852a011a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:09.615986 12566 raft_consensus.cc:729] T 641ac19be124433caea3225631bafd67 P 3f0c22abd96e44ed8d7b4616852a011a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f0c22abd96e44ed8d7b4616852a011a, State: Initialized, Role: FOLLOWER
I20260809 08:52:09.616004 12562 consensus_queue.cc:260] T 641ac19be124433caea3225631bafd67 P 30e8e1019603485185b0556dda0d8d37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } } peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } }
I20260809 08:52:09.616138 12566 consensus_queue.cc:260] T 641ac19be124433caea3225631bafd67 P 3f0c22abd96e44ed8d7b4616852a011a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } } peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } }
I20260809 08:52:09.616353 12562 ts_tablet_manager.cc:1428] T 641ac19be124433caea3225631bafd67 P 30e8e1019603485185b0556dda0d8d37: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:52:09.616365 12518 heartbeater.cc:503] Master 127.8.154.62:41637 was elected leader, sending a full tablet report...
I20260809 08:52:09.616436 12566 ts_tablet_manager.cc:1428] T 641ac19be124433caea3225631bafd67 P 3f0c22abd96e44ed8d7b4616852a011a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:52:09.616463 12565 tablet_bootstrap.cc:492] T 641ac19be124433caea3225631bafd67 P 106032240a5c4c63bc2816062da95e6f: No bootstrap required, opened a new log
I20260809 08:52:09.616520 12565 ts_tablet_manager.cc:1397] T 641ac19be124433caea3225631bafd67 P 106032240a5c4c63bc2816062da95e6f: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260809 08:52:09.616580 12039 heartbeater.cc:503] Master 127.8.154.62:41637 was elected leader, sending a full tablet report...
I20260809 08:52:09.618481 12565 raft_consensus.cc:348] T 641ac19be124433caea3225631bafd67 P 106032240a5c4c63bc2816062da95e6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } } peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } }
I20260809 08:52:09.618598 12565 raft_consensus.cc:374] T 641ac19be124433caea3225631bafd67 P 106032240a5c4c63bc2816062da95e6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:09.618618 12565 raft_consensus.cc:729] T 641ac19be124433caea3225631bafd67 P 106032240a5c4c63bc2816062da95e6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 106032240a5c4c63bc2816062da95e6f, State: Initialized, Role: FOLLOWER
I20260809 08:52:09.618724 12565 consensus_queue.cc:260] T 641ac19be124433caea3225631bafd67 P 106032240a5c4c63bc2816062da95e6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } } peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } }
I20260809 08:52:09.618947 12565 ts_tablet_manager.cc:1428] T 641ac19be124433caea3225631bafd67 P 106032240a5c4c63bc2816062da95e6f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 08:52:09.619029 12271 heartbeater.cc:503] Master 127.8.154.62:41637 was elected leader, sending a full tablet report...
W20260809 08:52:09.651397 12272 tablet.cc:2369] T 641ac19be124433caea3225631bafd67 P 106032240a5c4c63bc2816062da95e6f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 08:52:09.738575 12040 tablet.cc:2369] T 641ac19be124433caea3225631bafd67 P 3f0c22abd96e44ed8d7b4616852a011a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 08:52:09.818084 12519 tablet.cc:2369] T 641ac19be124433caea3225631bafd67 P 30e8e1019603485185b0556dda0d8d37: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:52:09.878011 12570 raft_consensus.cc:482] T 641ac19be124433caea3225631bafd67 P 30e8e1019603485185b0556dda0d8d37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:52:09.878163 12570 raft_consensus.cc:504] T 641ac19be124433caea3225631bafd67 P 30e8e1019603485185b0556dda0d8d37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } } peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } }
I20260809 08:52:09.878481 12570 leader_election.cc:288] T 641ac19be124433caea3225631bafd67 P 30e8e1019603485185b0556dda0d8d37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f0c22abd96e44ed8d7b4616852a011a (127.8.154.1:45991), 106032240a5c4c63bc2816062da95e6f (127.8.154.2:37729)
I20260809 08:52:09.882529 12561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:09.878831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:52297 (local address 127.8.154.2:37729)
0809 08:52:09.878881 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:09.878883 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:09.878920 (+ 37us) server_negotiation.cc:407] Connection header received
0809 08:52:09.878951 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:09.878952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:09.878964 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:09.879017 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:09.879208 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.879995 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.881522 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.882078 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.882153 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:09.882164 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:09.882417 (+ 253us) server_negotiation.cc:299] Negotiation successful
0809 08:52:09.882457 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 08:52:09.882529 12559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:09.878762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:49801 (local address 127.8.154.1:45991)
0809 08:52:09.878825 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:09.878827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:09.879109 (+ 282us) server_negotiation.cc:407] Connection header received
0809 08:52:09.879141 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:09.879142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:09.879151 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:09.879224 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:09.879365 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.880039 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.881522 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.882072 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.882153 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:09.882164 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:09.882417 (+ 253us) server_negotiation.cc:299] Negotiation successful
0809 08:52:09.882459 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 08:52:09.882547 12321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:09.878700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:37729 (local address 127.8.154.3:52297)
0809 08:52:09.878791 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:09.878807 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:09.878870 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:09.879041 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:09.879042 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:09.879045 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:09.879128 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:09.879131 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.880119 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.880120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:09.881425 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:09.881428 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.882219 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.882221 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:09.882314 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:09.882323 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 08:52:09.882394 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 08:52:09.882443 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":73}
I20260809 08:52:09.882557 12574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:09.878700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:45991 (local address 127.8.154.3:49801)
0809 08:52:09.878989 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:09.878997 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:09.879095 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:09.879242 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:09.879242 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:09.879243 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:09.879294 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:09.879295 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.880124 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.880124 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:09.881425 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:09.881428 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.882219 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.882221 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:09.882314 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:09.882323 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 08:52:09.882394 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 08:52:09.882441 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"mutex_wait_us":172,"thread_start_us":101,"threads_started":1}
I20260809 08:52:09.882975 12195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "641ac19be124433caea3225631bafd67" candidate_uuid: "30e8e1019603485185b0556dda0d8d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "106032240a5c4c63bc2816062da95e6f" is_pre_election: true
I20260809 08:52:09.883143 12195 raft_consensus.cc:2454] T 641ac19be124433caea3225631bafd67 P 106032240a5c4c63bc2816062da95e6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30e8e1019603485185b0556dda0d8d37 in term 0.
I20260809 08:52:09.883185 11971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "641ac19be124433caea3225631bafd67" candidate_uuid: "30e8e1019603485185b0556dda0d8d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f0c22abd96e44ed8d7b4616852a011a" is_pre_election: true
I20260809 08:52:09.883322 11971 raft_consensus.cc:2454] T 641ac19be124433caea3225631bafd67 P 3f0c22abd96e44ed8d7b4616852a011a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30e8e1019603485185b0556dda0d8d37 in term 0.
I20260809 08:52:09.883419 12312 leader_election.cc:302] T 641ac19be124433caea3225631bafd67 P 30e8e1019603485185b0556dda0d8d37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 106032240a5c4c63bc2816062da95e6f, 30e8e1019603485185b0556dda0d8d37; no voters:
I20260809 08:52:09.883595 12570 raft_consensus.cc:2781] T 641ac19be124433caea3225631bafd67 P 30e8e1019603485185b0556dda0d8d37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:52:09.883667 12570 raft_consensus.cc:482] T 641ac19be124433caea3225631bafd67 P 30e8e1019603485185b0556dda0d8d37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:52:09.883697 12570 raft_consensus.cc:3037] T 641ac19be124433caea3225631bafd67 P 30e8e1019603485185b0556dda0d8d37 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:09.884821 12570 raft_consensus.cc:504] T 641ac19be124433caea3225631bafd67 P 30e8e1019603485185b0556dda0d8d37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } } peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } }
I20260809 08:52:09.884970 12570 leader_election.cc:288] T 641ac19be124433caea3225631bafd67 P 30e8e1019603485185b0556dda0d8d37 [CANDIDATE]: Term 1 election: Requested vote from peers 3f0c22abd96e44ed8d7b4616852a011a (127.8.154.1:45991), 106032240a5c4c63bc2816062da95e6f (127.8.154.2:37729)
I20260809 08:52:09.885159 11971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "641ac19be124433caea3225631bafd67" candidate_uuid: "30e8e1019603485185b0556dda0d8d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f0c22abd96e44ed8d7b4616852a011a"
I20260809 08:52:09.885239 11971 raft_consensus.cc:3037] T 641ac19be124433caea3225631bafd67 P 3f0c22abd96e44ed8d7b4616852a011a [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:09.885875 11971 raft_consensus.cc:2454] T 641ac19be124433caea3225631bafd67 P 3f0c22abd96e44ed8d7b4616852a011a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30e8e1019603485185b0556dda0d8d37 in term 1.
I20260809 08:52:09.886001 12195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "641ac19be124433caea3225631bafd67" candidate_uuid: "30e8e1019603485185b0556dda0d8d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "106032240a5c4c63bc2816062da95e6f"
I20260809 08:52:09.886060 12311 leader_election.cc:302] T 641ac19be124433caea3225631bafd67 P 30e8e1019603485185b0556dda0d8d37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30e8e1019603485185b0556dda0d8d37, 3f0c22abd96e44ed8d7b4616852a011a; no voters:
I20260809 08:52:09.886103 12195 raft_consensus.cc:3037] T 641ac19be124433caea3225631bafd67 P 106032240a5c4c63bc2816062da95e6f [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:09.886154 12570 raft_consensus.cc:2781] T 641ac19be124433caea3225631bafd67 P 30e8e1019603485185b0556dda0d8d37 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:52:09.886309 12570 raft_consensus.cc:686] T 641ac19be124433caea3225631bafd67 P 30e8e1019603485185b0556dda0d8d37 [term 1 LEADER]: Becoming Leader. State: Replica: 30e8e1019603485185b0556dda0d8d37, State: Running, Role: LEADER
I20260809 08:52:09.886408 12570 consensus_queue.cc:237] T 641ac19be124433caea3225631bafd67 P 30e8e1019603485185b0556dda0d8d37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } } peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } }
I20260809 08:52:09.887019 12195 raft_consensus.cc:2454] T 641ac19be124433caea3225631bafd67 P 106032240a5c4c63bc2816062da95e6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30e8e1019603485185b0556dda0d8d37 in term 1.
I20260809 08:52:09.887264 11698 catalog_manager.cc:5495] T 641ac19be124433caea3225631bafd67 P 30e8e1019603485185b0556dda0d8d37 reported cstate change: term changed from 0 to 1, leader changed from <none> to 30e8e1019603485185b0556dda0d8d37 (127.8.154.3). New cstate: current_term: 1 leader_uuid: "30e8e1019603485185b0556dda0d8d37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } health_report { overall_health: UNKNOWN } } }
Aug 09 08:52:09 dist-test-slave-t3q3 krb5kdc[11636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265528, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:09.976148 11778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:09.968839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48424 (local address 127.8.154.62:41637)
0809 08:52:09.968909 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:09.968911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:09.969016 (+ 105us) server_negotiation.cc:407] Connection header received
0809 08:52:09.969063 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:09.969064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:09.969073 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:09.969129 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:09.969657 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.970175 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.970967 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.971148 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.971203 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:09.973354 (+ 2151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:09.973355 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:09.973355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:09.973361 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:09.974879 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:09.975271 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:09.975272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:09.975273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:09.975329 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:09.975539 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:09.975540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:09.975540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:09.975629 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:09.975674 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:09.975893 (+ 219us) server_negotiation.cc:299] Negotiation successful
0809 08:52:09.976026 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":46}
I20260809 08:52:09.976150 12587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:09.968781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:41637 (local address 127.0.0.1:48424)
0809 08:52:09.968939 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:09.968950 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:09.969018 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:09.969151 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:09.969152 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:09.969465 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:09.969586 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:09.969588 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.970266 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.970267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:09.970897 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:09.970899 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:09.971224 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:09.971225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:09.971271 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:09.971764 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:09.971768 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:09.973274 (+ 1506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:09.974952 (+ 1678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:09.974959 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:09.974961 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:09.975203 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:09.975432 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:09.975433 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:09.975433 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:09.975494 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:09.975700 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:09.975708 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:09.975763 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 08:52:09.975865 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 08:52:09.975977 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":1,"thread_start_us":102,"threads_started":1}
I20260809 08:52:09.978269 11698 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:48424:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 08:52:09.978477 11698 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:52:09.980099 12162 tablet_service.cc:1467] Processing CreateTablet for tablet 998279d324ef466cba06b760dba6405a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d544881d7c194d9690a01939573a8e9d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 08:52:09.980170 12391 tablet_service.cc:1467] Processing CreateTablet for tablet 998279d324ef466cba06b760dba6405a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d544881d7c194d9690a01939573a8e9d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 08:52:09.980161 11943 tablet_service.cc:1467] Processing CreateTablet for tablet 998279d324ef466cba06b760dba6405a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d544881d7c194d9690a01939573a8e9d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 08:52:09.980321 12391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 998279d324ef466cba06b760dba6405a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:09.980322 12162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 998279d324ef466cba06b760dba6405a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:09.980334 11943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 998279d324ef466cba06b760dba6405a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:09.981563 12566 tablet_bootstrap.cc:492] T 998279d324ef466cba06b760dba6405a P 3f0c22abd96e44ed8d7b4616852a011a: Bootstrap starting.
I20260809 08:52:09.981907 12565 tablet_bootstrap.cc:492] T 998279d324ef466cba06b760dba6405a P 106032240a5c4c63bc2816062da95e6f: Bootstrap starting.
I20260809 08:52:09.981913 12562 tablet_bootstrap.cc:492] T 998279d324ef466cba06b760dba6405a P 30e8e1019603485185b0556dda0d8d37: Bootstrap starting.
I20260809 08:52:09.981937 12566 tablet_bootstrap.cc:654] T 998279d324ef466cba06b760dba6405a P 3f0c22abd96e44ed8d7b4616852a011a: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:09.982228 12565 tablet_bootstrap.cc:654] T 998279d324ef466cba06b760dba6405a P 106032240a5c4c63bc2816062da95e6f: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:09.982363 12562 tablet_bootstrap.cc:654] T 998279d324ef466cba06b760dba6405a P 30e8e1019603485185b0556dda0d8d37: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:09.982795 12566 tablet_bootstrap.cc:492] T 998279d324ef466cba06b760dba6405a P 3f0c22abd96e44ed8d7b4616852a011a: No bootstrap required, opened a new log
I20260809 08:52:09.982849 12566 ts_tablet_manager.cc:1397] T 998279d324ef466cba06b760dba6405a P 3f0c22abd96e44ed8d7b4616852a011a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:09.983053 12566 raft_consensus.cc:348] T 998279d324ef466cba06b760dba6405a P 3f0c22abd96e44ed8d7b4616852a011a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } }
I20260809 08:52:09.983139 12566 raft_consensus.cc:374] T 998279d324ef466cba06b760dba6405a P 3f0c22abd96e44ed8d7b4616852a011a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:09.983162 12566 raft_consensus.cc:729] T 998279d324ef466cba06b760dba6405a P 3f0c22abd96e44ed8d7b4616852a011a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f0c22abd96e44ed8d7b4616852a011a, State: Initialized, Role: FOLLOWER
I20260809 08:52:09.983182 12562 tablet_bootstrap.cc:492] T 998279d324ef466cba06b760dba6405a P 30e8e1019603485185b0556dda0d8d37: No bootstrap required, opened a new log
I20260809 08:52:09.983229 12562 ts_tablet_manager.cc:1397] T 998279d324ef466cba06b760dba6405a P 30e8e1019603485185b0556dda0d8d37: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.001s
I20260809 08:52:09.983207 12566 consensus_queue.cc:260] T 998279d324ef466cba06b760dba6405a P 3f0c22abd96e44ed8d7b4616852a011a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } }
I20260809 08:52:09.983314 12566 ts_tablet_manager.cc:1428] T 998279d324ef466cba06b760dba6405a P 3f0c22abd96e44ed8d7b4616852a011a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 08:52:09.983376 12562 raft_consensus.cc:348] T 998279d324ef466cba06b760dba6405a P 30e8e1019603485185b0556dda0d8d37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } }
I20260809 08:52:09.983448 12562 raft_consensus.cc:374] T 998279d324ef466cba06b760dba6405a P 30e8e1019603485185b0556dda0d8d37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:09.983390 12571 tablet_replica.cc:439] T 998279d324ef466cba06b760dba6405a P 3f0c22abd96e44ed8d7b4616852a011a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } } }
I20260809 08:52:09.983474 12562 raft_consensus.cc:729] T 998279d324ef466cba06b760dba6405a P 30e8e1019603485185b0556dda0d8d37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30e8e1019603485185b0556dda0d8d37, State: Initialized, Role: FOLLOWER
I20260809 08:52:09.983491 12571 tablet_replica.cc:442] T 998279d324ef466cba06b760dba6405a P 3f0c22abd96e44ed8d7b4616852a011a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:52:09.983520 12562 consensus_queue.cc:260] T 998279d324ef466cba06b760dba6405a P 30e8e1019603485185b0556dda0d8d37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } }
I20260809 08:52:09.983636 12562 ts_tablet_manager.cc:1428] T 998279d324ef466cba06b760dba6405a P 30e8e1019603485185b0556dda0d8d37: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:09.983711 12570 tablet_replica.cc:439] T 998279d324ef466cba06b760dba6405a P 30e8e1019603485185b0556dda0d8d37: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } } }
I20260809 08:52:09.983789 12565 tablet_bootstrap.cc:492] T 998279d324ef466cba06b760dba6405a P 106032240a5c4c63bc2816062da95e6f: No bootstrap required, opened a new log
I20260809 08:52:09.983803 12570 tablet_replica.cc:442] T 998279d324ef466cba06b760dba6405a P 30e8e1019603485185b0556dda0d8d37: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:52:09.983834 12565 ts_tablet_manager.cc:1397] T 998279d324ef466cba06b760dba6405a P 106032240a5c4c63bc2816062da95e6f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:52:09.984005 12565 raft_consensus.cc:348] T 998279d324ef466cba06b760dba6405a P 106032240a5c4c63bc2816062da95e6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } }
I20260809 08:52:09.984099 12565 raft_consensus.cc:374] T 998279d324ef466cba06b760dba6405a P 106032240a5c4c63bc2816062da95e6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:09.984122 12565 raft_consensus.cc:729] T 998279d324ef466cba06b760dba6405a P 106032240a5c4c63bc2816062da95e6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 106032240a5c4c63bc2816062da95e6f, State: Initialized, Role: FOLLOWER
I20260809 08:52:09.984170 12565 consensus_queue.cc:260] T 998279d324ef466cba06b760dba6405a P 106032240a5c4c63bc2816062da95e6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } }
I20260809 08:52:09.984302 12565 ts_tablet_manager.cc:1428] T 998279d324ef466cba06b760dba6405a P 106032240a5c4c63bc2816062da95e6f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:09.984338 12573 tablet_replica.cc:439] T 998279d324ef466cba06b760dba6405a P 106032240a5c4c63bc2816062da95e6f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } } }
I20260809 08:52:09.984436 12573 tablet_replica.cc:442] T 998279d324ef466cba06b760dba6405a P 106032240a5c4c63bc2816062da95e6f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:52:10.017978 12570 raft_consensus.cc:482] T 998279d324ef466cba06b760dba6405a P 30e8e1019603485185b0556dda0d8d37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:52:10.018051 12570 raft_consensus.cc:504] T 998279d324ef466cba06b760dba6405a P 30e8e1019603485185b0556dda0d8d37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } }
I20260809 08:52:10.018190 12570 leader_election.cc:288] T 998279d324ef466cba06b760dba6405a P 30e8e1019603485185b0556dda0d8d37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f0c22abd96e44ed8d7b4616852a011a (127.8.154.1:45991), 106032240a5c4c63bc2816062da95e6f (127.8.154.2:37729)
I20260809 08:52:10.018366 11971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "998279d324ef466cba06b760dba6405a" candidate_uuid: "30e8e1019603485185b0556dda0d8d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f0c22abd96e44ed8d7b4616852a011a" is_pre_election: true
I20260809 08:52:10.018412 12195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "998279d324ef466cba06b760dba6405a" candidate_uuid: "30e8e1019603485185b0556dda0d8d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "106032240a5c4c63bc2816062da95e6f" is_pre_election: true
I20260809 08:52:10.018497 11971 raft_consensus.cc:2454] T 998279d324ef466cba06b760dba6405a P 3f0c22abd96e44ed8d7b4616852a011a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30e8e1019603485185b0556dda0d8d37 in term 0.
I20260809 08:52:10.018494 12195 raft_consensus.cc:2454] T 998279d324ef466cba06b760dba6405a P 106032240a5c4c63bc2816062da95e6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30e8e1019603485185b0556dda0d8d37 in term 0.
I20260809 08:52:10.018674 12312 leader_election.cc:302] T 998279d324ef466cba06b760dba6405a P 30e8e1019603485185b0556dda0d8d37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 106032240a5c4c63bc2816062da95e6f, 30e8e1019603485185b0556dda0d8d37; no voters:
I20260809 08:52:10.018822 12570 raft_consensus.cc:2781] T 998279d324ef466cba06b760dba6405a P 30e8e1019603485185b0556dda0d8d37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:52:10.018879 12570 raft_consensus.cc:482] T 998279d324ef466cba06b760dba6405a P 30e8e1019603485185b0556dda0d8d37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:52:10.018895 12570 raft_consensus.cc:3037] T 998279d324ef466cba06b760dba6405a P 30e8e1019603485185b0556dda0d8d37 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:10.019526 12570 raft_consensus.cc:504] T 998279d324ef466cba06b760dba6405a P 30e8e1019603485185b0556dda0d8d37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } }
I20260809 08:52:10.019680 12570 leader_election.cc:288] T 998279d324ef466cba06b760dba6405a P 30e8e1019603485185b0556dda0d8d37 [CANDIDATE]: Term 1 election: Requested vote from peers 3f0c22abd96e44ed8d7b4616852a011a (127.8.154.1:45991), 106032240a5c4c63bc2816062da95e6f (127.8.154.2:37729)
I20260809 08:52:10.019827 11971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "998279d324ef466cba06b760dba6405a" candidate_uuid: "30e8e1019603485185b0556dda0d8d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f0c22abd96e44ed8d7b4616852a011a"
I20260809 08:52:10.019860 12195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "998279d324ef466cba06b760dba6405a" candidate_uuid: "30e8e1019603485185b0556dda0d8d37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "106032240a5c4c63bc2816062da95e6f"
I20260809 08:52:10.019913 11971 raft_consensus.cc:3037] T 998279d324ef466cba06b760dba6405a P 3f0c22abd96e44ed8d7b4616852a011a [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:10.019922 12195 raft_consensus.cc:3037] T 998279d324ef466cba06b760dba6405a P 106032240a5c4c63bc2816062da95e6f [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:10.020601 12195 raft_consensus.cc:2454] T 998279d324ef466cba06b760dba6405a P 106032240a5c4c63bc2816062da95e6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30e8e1019603485185b0556dda0d8d37 in term 1.
I20260809 08:52:10.020604 11971 raft_consensus.cc:2454] T 998279d324ef466cba06b760dba6405a P 3f0c22abd96e44ed8d7b4616852a011a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30e8e1019603485185b0556dda0d8d37 in term 1.
I20260809 08:52:10.020784 12312 leader_election.cc:302] T 998279d324ef466cba06b760dba6405a P 30e8e1019603485185b0556dda0d8d37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 106032240a5c4c63bc2816062da95e6f, 30e8e1019603485185b0556dda0d8d37; no voters:
I20260809 08:52:10.020954 12570 raft_consensus.cc:2781] T 998279d324ef466cba06b760dba6405a P 30e8e1019603485185b0556dda0d8d37 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:52:10.021026 12570 raft_consensus.cc:686] T 998279d324ef466cba06b760dba6405a P 30e8e1019603485185b0556dda0d8d37 [term 1 LEADER]: Becoming Leader. State: Replica: 30e8e1019603485185b0556dda0d8d37, State: Running, Role: LEADER
I20260809 08:52:10.021061 12570 consensus_queue.cc:237] T 998279d324ef466cba06b760dba6405a P 30e8e1019603485185b0556dda0d8d37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } }
I20260809 08:52:10.021461 12577 tablet_replica.cc:439] T 998279d324ef466cba06b760dba6405a P 30e8e1019603485185b0556dda0d8d37: TxnStatusTablet state changed. Reason: New leader 30e8e1019603485185b0556dda0d8d37. Latest consensus state: current_term: 1 leader_uuid: "30e8e1019603485185b0556dda0d8d37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } } }
I20260809 08:52:10.021580 12577 tablet_replica.cc:442] T 998279d324ef466cba06b760dba6405a P 30e8e1019603485185b0556dda0d8d37: This TxnStatusTablet replica's current role is: LEADER
I20260809 08:52:10.021835 12596 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 08:52:10.021761 11698 catalog_manager.cc:5495] T 998279d324ef466cba06b760dba6405a P 30e8e1019603485185b0556dda0d8d37 reported cstate change: term changed from 0 to 1, leader changed from <none> to 30e8e1019603485185b0556dda0d8d37 (127.8.154.3). New cstate: current_term: 1 leader_uuid: "30e8e1019603485185b0556dda0d8d37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } health_report { overall_health: HEALTHY } } }
I20260809 08:52:10.021966 12596 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 08:52:10.031337 12558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:10.028597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33114 (local address 127.8.154.3:43665)
0809 08:52:10.028650 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:10.028652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:10.028662 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:52:10.028760 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:10.028760 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:10.028776 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:10.028836 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:52:10.028992 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:10.029478 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:10.030596 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:10.030780 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:10.030825 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:10.031044 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:52:10.031131 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:52:10.031188 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:10.031271 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 08:52:10.031289 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 08:52:10.031407 12587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:10.028527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:43665 (local address 127.0.0.1:33114)
0809 08:52:10.028615 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:10.028627 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:10.028674 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:10.028866 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:10.028867 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:10.028868 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 08:52:10.028923 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:10.028925 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:10.029563 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:10.029564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:10.030505 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:10.030509 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:10.030852 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:10.030853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:10.030926 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:10.030945 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 08:52:10.031199 (+ 254us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 08:52:10.031208 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 08:52:10.031269 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 08:52:10.031328 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 08:52:10.033114 12195 raft_consensus.cc:1261] T 998279d324ef466cba06b760dba6405a P 106032240a5c4c63bc2816062da95e6f [term 1 FOLLOWER]: Refusing update from remote peer 30e8e1019603485185b0556dda0d8d37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:52:10.033181 11971 raft_consensus.cc:1261] T 998279d324ef466cba06b760dba6405a P 3f0c22abd96e44ed8d7b4616852a011a [term 1 FOLLOWER]: Refusing update from remote peer 30e8e1019603485185b0556dda0d8d37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:52:10.033483 12570 consensus_queue.cc:1035] T 998279d324ef466cba06b760dba6405a P 30e8e1019603485185b0556dda0d8d37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:52:10.033613 12577 consensus_queue.cc:1035] T 998279d324ef466cba06b760dba6405a P 30e8e1019603485185b0556dda0d8d37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:52:10.034725 12571 tablet_replica.cc:439] T 998279d324ef466cba06b760dba6405a P 3f0c22abd96e44ed8d7b4616852a011a: TxnStatusTablet state changed. Reason: New leader 30e8e1019603485185b0556dda0d8d37. Latest consensus state: current_term: 1 leader_uuid: "30e8e1019603485185b0556dda0d8d37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } } }
I20260809 08:52:10.034816 12571 tablet_replica.cc:442] T 998279d324ef466cba06b760dba6405a P 3f0c22abd96e44ed8d7b4616852a011a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:52:10.034832 12573 tablet_replica.cc:439] T 998279d324ef466cba06b760dba6405a P 106032240a5c4c63bc2816062da95e6f: TxnStatusTablet state changed. Reason: New leader 30e8e1019603485185b0556dda0d8d37. Latest consensus state: current_term: 1 leader_uuid: "30e8e1019603485185b0556dda0d8d37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } } }
I20260809 08:52:10.034925 12573 tablet_replica.cc:442] T 998279d324ef466cba06b760dba6405a P 106032240a5c4c63bc2816062da95e6f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:52:10.036154 12577 tablet_replica.cc:439] T 998279d324ef466cba06b760dba6405a P 30e8e1019603485185b0556dda0d8d37: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "30e8e1019603485185b0556dda0d8d37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } } }
I20260809 08:52:10.036254 12577 tablet_replica.cc:442] T 998279d324ef466cba06b760dba6405a P 30e8e1019603485185b0556dda0d8d37: This TxnStatusTablet replica's current role is: LEADER
I20260809 08:52:10.036326 12609 mvcc.cc:205] Tried to move back new op lower bound from 7316543611013365760 to 7316543610966831104. Current Snapshot: MvccSnapshot[applied={T|T < 7316543611013365760}]
I20260809 08:52:10.036448 12571 tablet_replica.cc:439] T 998279d324ef466cba06b760dba6405a P 3f0c22abd96e44ed8d7b4616852a011a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "30e8e1019603485185b0556dda0d8d37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } } }
I20260809 08:52:10.036523 12571 tablet_replica.cc:442] T 998279d324ef466cba06b760dba6405a P 3f0c22abd96e44ed8d7b4616852a011a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:52:10.036763 12610 mvcc.cc:205] Tried to move back new op lower bound from 7316543611013365760 to 7316543610966831104. Current Snapshot: MvccSnapshot[applied={T|T < 7316543611013365760}]
I20260809 08:52:10.037128 12570 tablet_replica.cc:439] T 998279d324ef466cba06b760dba6405a P 30e8e1019603485185b0556dda0d8d37: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "30e8e1019603485185b0556dda0d8d37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } } }
I20260809 08:52:10.037210 12570 tablet_replica.cc:442] T 998279d324ef466cba06b760dba6405a P 30e8e1019603485185b0556dda0d8d37: This TxnStatusTablet replica's current role is: LEADER
I20260809 08:52:10.037518 12573 tablet_replica.cc:439] T 998279d324ef466cba06b760dba6405a P 106032240a5c4c63bc2816062da95e6f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "30e8e1019603485185b0556dda0d8d37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } } }
I20260809 08:52:10.037602 12573 tablet_replica.cc:442] T 998279d324ef466cba06b760dba6405a P 106032240a5c4c63bc2816062da95e6f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:52:10.044632 12558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:10.042135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33118 (local address 127.8.154.3:43665)
0809 08:52:10.042179 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:10.042181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:10.042190 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:52:10.042233 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:10.042234 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:10.042243 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:10.042308 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:52:10.042388 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:10.043160 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:10.043896 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:10.044226 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:10.044296 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:10.044421 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:52:10.044456 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:52:10.044512 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:10.044546 (+ 34us) server_negotiation.cc:299] Negotiation successful
0809 08:52:10.044565 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 08:52:10.049657 12528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:10.047049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:43665 (local address 127.8.154.3:37375)
0809 08:52:10.047117 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:10.047128 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:10.047185 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:10.047324 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:10.047325 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:10.047326 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 08:52:10.047386 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:10.047387 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:10.048252 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:10.048252 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:10.048803 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:10.048804 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:10.049272 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:10.049273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:10.049325 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:10.049338 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 08:52:10.049521 (+ 183us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 08:52:10.049522 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 08:52:10.049557 (+ 35us) client_negotiation.cc:240] Negotiation successful
0809 08:52:10.049590 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 08:52:10.049671 12558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:10.047104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:37375 (local address 127.8.154.3:43665)
0809 08:52:10.047142 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:10.047144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:10.047169 (+ 25us) server_negotiation.cc:407] Connection header received
0809 08:52:10.047248 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:10.047248 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:10.047256 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:10.047308 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:52:10.047454 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:10.048174 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:10.048885 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:10.049188 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:10.049260 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:10.049425 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:52:10.049456 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:52:10.049509 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:10.049562 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 08:52:10.049580 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 08:52:10.051915 12195 raft_consensus.cc:1261] T 641ac19be124433caea3225631bafd67 P 106032240a5c4c63bc2816062da95e6f [term 1 FOLLOWER]: Refusing update from remote peer 30e8e1019603485185b0556dda0d8d37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:52:10.051970 11971 raft_consensus.cc:1261] T 641ac19be124433caea3225631bafd67 P 3f0c22abd96e44ed8d7b4616852a011a [term 1 FOLLOWER]: Refusing update from remote peer 30e8e1019603485185b0556dda0d8d37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:52:10.052134 12577 consensus_queue.cc:1035] T 641ac19be124433caea3225631bafd67 P 30e8e1019603485185b0556dda0d8d37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:52:10.052301 12577 consensus_queue.cc:1035] T 641ac19be124433caea3225631bafd67 P 30e8e1019603485185b0556dda0d8d37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:52:10.054906 12611 mvcc.cc:205] Tried to move back new op lower bound from 7316543611091001344 to 7316543610415284224. Current Snapshot: MvccSnapshot[applied={T|T < 7316543611091001344}]
I20260809 08:52:10.066309 11696 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48414:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:52:10.066421 11696 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48414:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:52:10.067153 11696 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 394558cf3144481bbebd8079b686db41: Sending DeleteTablet for 3 replicas of tablet 641ac19be124433caea3225631bafd67
I20260809 08:52:10.067471 12162 tablet_service.cc:1514] Processing DeleteTablet for tablet 641ac19be124433caea3225631bafd67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:10 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:52658
I20260809 08:52:10.067471 11943 tablet_service.cc:1514] Processing DeleteTablet for tablet 641ac19be124433caea3225631bafd67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:10 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:54252
I20260809 08:52:10.067646 11696 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48414:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:52:10.067792 11696 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:52:10.067770 12391 tablet_service.cc:1514] Processing DeleteTablet for tablet 641ac19be124433caea3225631bafd67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:10 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:33100
I20260809 08:52:10.068014 12636 tablet_replica.cc:330] T 641ac19be124433caea3225631bafd67 P 30e8e1019603485185b0556dda0d8d37: stopping tablet replica
I20260809 08:52:10.068199 12636 raft_consensus.cc:2229] T 641ac19be124433caea3225631bafd67 P 30e8e1019603485185b0556dda0d8d37 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:52:10.068250 12634 tablet_replica.cc:330] T 641ac19be124433caea3225631bafd67 P 106032240a5c4c63bc2816062da95e6f: stopping tablet replica
I20260809 08:52:10.068343 12634 raft_consensus.cc:2229] T 641ac19be124433caea3225631bafd67 P 106032240a5c4c63bc2816062da95e6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:52:10.068405 12634 raft_consensus.cc:2258] T 641ac19be124433caea3225631bafd67 P 106032240a5c4c63bc2816062da95e6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:52:10.068485 12636 raft_consensus.cc:2258] T 641ac19be124433caea3225631bafd67 P 30e8e1019603485185b0556dda0d8d37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:52:10.068868 11943 tablet_service.cc:1467] Processing CreateTablet for tablet 78374e55424442b99800c120ca55f2b2 (DEFAULT_TABLE table=test-table [id=d2f9dbdcfac7499db3e873593225b17f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:10.068892 12162 tablet_service.cc:1467] Processing CreateTablet for tablet 78374e55424442b99800c120ca55f2b2 (DEFAULT_TABLE table=test-table [id=d2f9dbdcfac7499db3e873593225b17f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:10.068996 11943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78374e55424442b99800c120ca55f2b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:10.068995 12162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78374e55424442b99800c120ca55f2b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:10.068987 12391 tablet_service.cc:1467] Processing CreateTablet for tablet 78374e55424442b99800c120ca55f2b2 (DEFAULT_TABLE table=test-table [id=d2f9dbdcfac7499db3e873593225b17f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:10.069115 12391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78374e55424442b99800c120ca55f2b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:10.070410 12565 tablet_bootstrap.cc:492] T 78374e55424442b99800c120ca55f2b2 P 106032240a5c4c63bc2816062da95e6f: Bootstrap starting.
I20260809 08:52:10.070453 12566 tablet_bootstrap.cc:492] T 78374e55424442b99800c120ca55f2b2 P 3f0c22abd96e44ed8d7b4616852a011a: Bootstrap starting.
I20260809 08:52:10.070443 12562 tablet_bootstrap.cc:492] T 78374e55424442b99800c120ca55f2b2 P 30e8e1019603485185b0556dda0d8d37: Bootstrap starting.
I20260809 08:52:10.070916 12565 tablet_bootstrap.cc:654] T 78374e55424442b99800c120ca55f2b2 P 106032240a5c4c63bc2816062da95e6f: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:10.070931 12562 tablet_bootstrap.cc:654] T 78374e55424442b99800c120ca55f2b2 P 30e8e1019603485185b0556dda0d8d37: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:10.071139 12566 tablet_bootstrap.cc:654] T 78374e55424442b99800c120ca55f2b2 P 3f0c22abd96e44ed8d7b4616852a011a: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:10.071422 12634 ts_tablet_manager.cc:1905] T 641ac19be124433caea3225631bafd67 P 106032240a5c4c63bc2816062da95e6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:52:10.071563 12562 tablet_bootstrap.cc:492] T 78374e55424442b99800c120ca55f2b2 P 30e8e1019603485185b0556dda0d8d37: No bootstrap required, opened a new log
I20260809 08:52:10.071619 12562 ts_tablet_manager.cc:1397] T 78374e55424442b99800c120ca55f2b2 P 30e8e1019603485185b0556dda0d8d37: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260809 08:52:10.071764 12562 raft_consensus.cc:348] T 78374e55424442b99800c120ca55f2b2 P 30e8e1019603485185b0556dda0d8d37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } }
I20260809 08:52:10.071826 12562 raft_consensus.cc:374] T 78374e55424442b99800c120ca55f2b2 P 30e8e1019603485185b0556dda0d8d37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:10.071848 12562 raft_consensus.cc:729] T 78374e55424442b99800c120ca55f2b2 P 30e8e1019603485185b0556dda0d8d37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30e8e1019603485185b0556dda0d8d37, State: Initialized, Role: FOLLOWER
I20260809 08:52:10.071890 12562 consensus_queue.cc:260] T 78374e55424442b99800c120ca55f2b2 P 30e8e1019603485185b0556dda0d8d37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } }
I20260809 08:52:10.071983 12562 ts_tablet_manager.cc:1428] T 78374e55424442b99800c120ca55f2b2 P 30e8e1019603485185b0556dda0d8d37: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:10.072360 12636 ts_tablet_manager.cc:1905] T 641ac19be124433caea3225631bafd67 P 30e8e1019603485185b0556dda0d8d37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:52:10.073773 12635 tablet_replica.cc:330] T 641ac19be124433caea3225631bafd67 P 3f0c22abd96e44ed8d7b4616852a011a: stopping tablet replica
I20260809 08:52:10.073836 12634 ts_tablet_manager.cc:1918] T 641ac19be124433caea3225631bafd67 P 106032240a5c4c63bc2816062da95e6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:52:10.073897 12635 raft_consensus.cc:2229] T 641ac19be124433caea3225631bafd67 P 3f0c22abd96e44ed8d7b4616852a011a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:52:10.073951 12634 log.cc:1196] T 641ac19be124433caea3225631bafd67 P 106032240a5c4c63bc2816062da95e6f: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-1/wal/wals/641ac19be124433caea3225631bafd67
I20260809 08:52:10.073971 12635 raft_consensus.cc:2258] T 641ac19be124433caea3225631bafd67 P 3f0c22abd96e44ed8d7b4616852a011a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:52:10.074126 12636 ts_tablet_manager.cc:1918] T 641ac19be124433caea3225631bafd67 P 30e8e1019603485185b0556dda0d8d37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:52:10.074206 12636 log.cc:1196] T 641ac19be124433caea3225631bafd67 P 30e8e1019603485185b0556dda0d8d37: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-2/wal/wals/641ac19be124433caea3225631bafd67
I20260809 08:52:10.074484 12636 ts_tablet_manager.cc:1939] T 641ac19be124433caea3225631bafd67 P 30e8e1019603485185b0556dda0d8d37: Deleting consensus metadata
I20260809 08:52:10.074486 12635 ts_tablet_manager.cc:1905] T 641ac19be124433caea3225631bafd67 P 3f0c22abd96e44ed8d7b4616852a011a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:52:10.074561 12634 ts_tablet_manager.cc:1939] T 641ac19be124433caea3225631bafd67 P 106032240a5c4c63bc2816062da95e6f: Deleting consensus metadata
I20260809 08:52:10.074891 11679 catalog_manager.cc:4841] TS 30e8e1019603485185b0556dda0d8d37 (127.8.154.3:43665): tablet 641ac19be124433caea3225631bafd67 (table test-table [id=130e86c8a0b94ffd8091bfb037f020cf]) successfully deleted
I20260809 08:52:10.075049 11679 catalog_manager.cc:4841] TS 106032240a5c4c63bc2816062da95e6f (127.8.154.2:37729): tablet 641ac19be124433caea3225631bafd67 (table test-table [id=130e86c8a0b94ffd8091bfb037f020cf]) successfully deleted
I20260809 08:52:10.075281 12566 tablet_bootstrap.cc:492] T 78374e55424442b99800c120ca55f2b2 P 3f0c22abd96e44ed8d7b4616852a011a: No bootstrap required, opened a new log
I20260809 08:52:10.075335 12566 ts_tablet_manager.cc:1397] T 78374e55424442b99800c120ca55f2b2 P 3f0c22abd96e44ed8d7b4616852a011a: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260809 08:52:10.075466 12566 raft_consensus.cc:348] T 78374e55424442b99800c120ca55f2b2 P 3f0c22abd96e44ed8d7b4616852a011a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } }
I20260809 08:52:10.075539 12566 raft_consensus.cc:374] T 78374e55424442b99800c120ca55f2b2 P 3f0c22abd96e44ed8d7b4616852a011a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:10.075567 12566 raft_consensus.cc:729] T 78374e55424442b99800c120ca55f2b2 P 3f0c22abd96e44ed8d7b4616852a011a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f0c22abd96e44ed8d7b4616852a011a, State: Initialized, Role: FOLLOWER
I20260809 08:52:10.075619 12566 consensus_queue.cc:260] T 78374e55424442b99800c120ca55f2b2 P 3f0c22abd96e44ed8d7b4616852a011a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } }
I20260809 08:52:10.075730 12566 ts_tablet_manager.cc:1428] T 78374e55424442b99800c120ca55f2b2 P 3f0c22abd96e44ed8d7b4616852a011a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 08:52:10.076210 12565 tablet_bootstrap.cc:492] T 78374e55424442b99800c120ca55f2b2 P 106032240a5c4c63bc2816062da95e6f: No bootstrap required, opened a new log
I20260809 08:52:10.076264 12565 ts_tablet_manager.cc:1397] T 78374e55424442b99800c120ca55f2b2 P 106032240a5c4c63bc2816062da95e6f: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260809 08:52:10.076421 12635 ts_tablet_manager.cc:1918] T 641ac19be124433caea3225631bafd67 P 3f0c22abd96e44ed8d7b4616852a011a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:52:10.076421 12565 raft_consensus.cc:348] T 78374e55424442b99800c120ca55f2b2 P 106032240a5c4c63bc2816062da95e6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } }
I20260809 08:52:10.076491 12635 log.cc:1196] T 641ac19be124433caea3225631bafd67 P 3f0c22abd96e44ed8d7b4616852a011a: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-0/wal/wals/641ac19be124433caea3225631bafd67
I20260809 08:52:10.076493 12565 raft_consensus.cc:374] T 78374e55424442b99800c120ca55f2b2 P 106032240a5c4c63bc2816062da95e6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:10.076515 12565 raft_consensus.cc:729] T 78374e55424442b99800c120ca55f2b2 P 106032240a5c4c63bc2816062da95e6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 106032240a5c4c63bc2816062da95e6f, State: Initialized, Role: FOLLOWER
I20260809 08:52:10.076562 12565 consensus_queue.cc:260] T 78374e55424442b99800c120ca55f2b2 P 106032240a5c4c63bc2816062da95e6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } }
I20260809 08:52:10.076663 12565 ts_tablet_manager.cc:1428] T 78374e55424442b99800c120ca55f2b2 P 106032240a5c4c63bc2816062da95e6f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:10.076787 12635 ts_tablet_manager.cc:1939] T 641ac19be124433caea3225631bafd67 P 3f0c22abd96e44ed8d7b4616852a011a: Deleting consensus metadata
I20260809 08:52:10.077193 11678 catalog_manager.cc:4841] TS 3f0c22abd96e44ed8d7b4616852a011a (127.8.154.1:45991): tablet 641ac19be124433caea3225631bafd67 (table test-table [id=130e86c8a0b94ffd8091bfb037f020cf]) successfully deleted
I20260809 08:52:10.352213 12571 raft_consensus.cc:482] T 78374e55424442b99800c120ca55f2b2 P 3f0c22abd96e44ed8d7b4616852a011a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:52:10.352350 12571 raft_consensus.cc:504] T 78374e55424442b99800c120ca55f2b2 P 3f0c22abd96e44ed8d7b4616852a011a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } }
I20260809 08:52:10.352816 12571 leader_election.cc:288] T 78374e55424442b99800c120ca55f2b2 P 3f0c22abd96e44ed8d7b4616852a011a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 106032240a5c4c63bc2816062da95e6f (127.8.154.2:37729), 30e8e1019603485185b0556dda0d8d37 (127.8.154.3:43665)
I20260809 08:52:10.356949 12558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:10.353443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:49449 (local address 127.8.154.3:43665)
0809 08:52:10.353544 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:10.353547 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:10.353557 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:52:10.353587 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:10.353588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:10.353597 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:10.353671 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:10.353815 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:10.354577 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:10.356111 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:10.356587 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:10.356675 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:10.356689 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:10.356842 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 08:52:10.356882 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 08:52:10.356963 12645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:10.353352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:43665 (local address 127.8.154.1:49449)
0809 08:52:10.353443 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:10.353453 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:10.353518 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:10.353685 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:10.353686 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:10.353687 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:10.353732 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:10.353734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:10.354686 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:10.354688 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:10.355946 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:10.355949 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:10.356676 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:10.356677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:10.356759 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:10.356763 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:52:10.356833 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 08:52:10.356875 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76,"mutex_wait_us":220,"thread_start_us":56,"threads_started":1}
I20260809 08:52:10.357074 12561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:10.353228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:50799 (local address 127.8.154.2:37729)
0809 08:52:10.353299 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:10.353301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:10.353373 (+ 72us) server_negotiation.cc:407] Connection header received
0809 08:52:10.353510 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:10.353511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:10.353524 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:10.353561 (+ 37us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:10.353819 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:10.354588 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:10.356117 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:10.356673 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:10.356763 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:10.356775 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:10.356953 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 08:52:10.356997 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 08:52:10.357100 12644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:10.353109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:37729 (local address 127.8.154.1:50799)
0809 08:52:10.353288 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:10.353304 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:10.353412 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:10.353650 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:10.353652 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:10.353654 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:10.353725 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:10.353727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:10.354716 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:10.354717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:10.355958 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:10.355961 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:10.356780 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:10.356781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:10.356867 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:10.356871 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:52:10.356946 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 08:52:10.357000 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":97,"thread_start_us":120,"threads_started":1}
I20260809 08:52:10.357250 12435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78374e55424442b99800c120ca55f2b2" candidate_uuid: "3f0c22abd96e44ed8d7b4616852a011a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30e8e1019603485185b0556dda0d8d37" is_pre_election: true
I20260809 08:52:10.357326 12195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78374e55424442b99800c120ca55f2b2" candidate_uuid: "3f0c22abd96e44ed8d7b4616852a011a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "106032240a5c4c63bc2816062da95e6f" is_pre_election: true
I20260809 08:52:10.357406 12195 raft_consensus.cc:2454] T 78374e55424442b99800c120ca55f2b2 P 106032240a5c4c63bc2816062da95e6f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f0c22abd96e44ed8d7b4616852a011a in term 0.
I20260809 08:52:10.357416 12435 raft_consensus.cc:2454] T 78374e55424442b99800c120ca55f2b2 P 30e8e1019603485185b0556dda0d8d37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f0c22abd96e44ed8d7b4616852a011a in term 0.
I20260809 08:52:10.357609 11857 leader_election.cc:302] T 78374e55424442b99800c120ca55f2b2 P 3f0c22abd96e44ed8d7b4616852a011a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 106032240a5c4c63bc2816062da95e6f, 3f0c22abd96e44ed8d7b4616852a011a; no voters:
I20260809 08:52:10.357760 12571 raft_consensus.cc:2781] T 78374e55424442b99800c120ca55f2b2 P 3f0c22abd96e44ed8d7b4616852a011a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:52:10.357816 12571 raft_consensus.cc:482] T 78374e55424442b99800c120ca55f2b2 P 3f0c22abd96e44ed8d7b4616852a011a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:52:10.357842 12571 raft_consensus.cc:3037] T 78374e55424442b99800c120ca55f2b2 P 3f0c22abd96e44ed8d7b4616852a011a [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:10.358558 12571 raft_consensus.cc:504] T 78374e55424442b99800c120ca55f2b2 P 3f0c22abd96e44ed8d7b4616852a011a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } }
I20260809 08:52:10.358705 12571 leader_election.cc:288] T 78374e55424442b99800c120ca55f2b2 P 3f0c22abd96e44ed8d7b4616852a011a [CANDIDATE]: Term 1 election: Requested vote from peers 106032240a5c4c63bc2816062da95e6f (127.8.154.2:37729), 30e8e1019603485185b0556dda0d8d37 (127.8.154.3:43665)
I20260809 08:52:10.358892 12195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78374e55424442b99800c120ca55f2b2" candidate_uuid: "3f0c22abd96e44ed8d7b4616852a011a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "106032240a5c4c63bc2816062da95e6f"
I20260809 08:52:10.358929 12435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78374e55424442b99800c120ca55f2b2" candidate_uuid: "3f0c22abd96e44ed8d7b4616852a011a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30e8e1019603485185b0556dda0d8d37"
I20260809 08:52:10.358982 12195 raft_consensus.cc:3037] T 78374e55424442b99800c120ca55f2b2 P 106032240a5c4c63bc2816062da95e6f [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:10.359007 12435 raft_consensus.cc:3037] T 78374e55424442b99800c120ca55f2b2 P 30e8e1019603485185b0556dda0d8d37 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:10.359828 12195 raft_consensus.cc:2454] T 78374e55424442b99800c120ca55f2b2 P 106032240a5c4c63bc2816062da95e6f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f0c22abd96e44ed8d7b4616852a011a in term 1.
I20260809 08:52:10.359828 12435 raft_consensus.cc:2454] T 78374e55424442b99800c120ca55f2b2 P 30e8e1019603485185b0556dda0d8d37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f0c22abd96e44ed8d7b4616852a011a in term 1.
I20260809 08:52:10.360008 11857 leader_election.cc:302] T 78374e55424442b99800c120ca55f2b2 P 3f0c22abd96e44ed8d7b4616852a011a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30e8e1019603485185b0556dda0d8d37, 3f0c22abd96e44ed8d7b4616852a011a; no voters:
I20260809 08:52:10.360152 12571 raft_consensus.cc:2781] T 78374e55424442b99800c120ca55f2b2 P 3f0c22abd96e44ed8d7b4616852a011a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:52:10.360323 12571 raft_consensus.cc:686] T 78374e55424442b99800c120ca55f2b2 P 3f0c22abd96e44ed8d7b4616852a011a [term 1 LEADER]: Becoming Leader. State: Replica: 3f0c22abd96e44ed8d7b4616852a011a, State: Running, Role: LEADER
I20260809 08:52:10.360385 12571 consensus_queue.cc:237] T 78374e55424442b99800c120ca55f2b2 P 3f0c22abd96e44ed8d7b4616852a011a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } }
I20260809 08:52:10.361235 11698 catalog_manager.cc:5495] T 78374e55424442b99800c120ca55f2b2 P 3f0c22abd96e44ed8d7b4616852a011a reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f0c22abd96e44ed8d7b4616852a011a (127.8.154.1). New cstate: current_term: 1 leader_uuid: "3f0c22abd96e44ed8d7b4616852a011a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } health_report { overall_health: UNKNOWN } } }
I20260809 08:52:10.430058 12651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:10.427515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54266 (local address 127.8.154.1:45991)
0809 08:52:10.427658 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:10.427660 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:10.427667 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:52:10.427710 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:10.427711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:10.427735 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:10.427830 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:52:10.427944 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:10.428457 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:10.429200 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:10.429425 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:10.429506 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:10.429676 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:52:10.429804 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:52:10.429894 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:10.429939 (+ 45us) server_negotiation.cc:299] Negotiation successful
0809 08:52:10.429961 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":73,"threads_started":1}
I20260809 08:52:10.435329 12558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:10.432830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:60613 (local address 127.8.154.3:43665)
0809 08:52:10.432900 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:10.432902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:10.432938 (+ 36us) server_negotiation.cc:407] Connection header received
0809 08:52:10.433023 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:10.433023 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:10.433033 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:10.433092 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:52:10.433463 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:10.433964 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:10.434698 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:10.434891 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:10.434946 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:10.435113 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:52:10.435140 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:52:10.435183 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:10.435256 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 08:52:10.435272 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 08:52:10.435359 12660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:10.432721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:43665 (local address 127.8.154.1:60613)
0809 08:52:10.432876 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:10.432887 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:10.432947 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:10.433093 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:10.433094 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:10.433095 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 08:52:10.433393 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:10.433395 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:10.434059 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:10.434060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:10.434622 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:10.434624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:10.434978 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:10.434979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:10.435025 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:10.435039 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 08:52:10.435200 (+ 161us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 08:52:10.435201 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 08:52:10.435245 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 08:52:10.435279 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":85,"threads_started":1}
I20260809 08:52:10.437532 12195 raft_consensus.cc:1261] T 78374e55424442b99800c120ca55f2b2 P 106032240a5c4c63bc2816062da95e6f [term 1 FOLLOWER]: Refusing update from remote peer 3f0c22abd96e44ed8d7b4616852a011a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:52:10.437639 12435 raft_consensus.cc:1261] T 78374e55424442b99800c120ca55f2b2 P 30e8e1019603485185b0556dda0d8d37 [term 1 FOLLOWER]: Refusing update from remote peer 3f0c22abd96e44ed8d7b4616852a011a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:52:10.437844 12646 consensus_queue.cc:1035] T 78374e55424442b99800c120ca55f2b2 P 3f0c22abd96e44ed8d7b4616852a011a [LEADER]: Connected to new peer: Peer: permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:52:10.437961 12571 consensus_queue.cc:1035] T 78374e55424442b99800c120ca55f2b2 P 3f0c22abd96e44ed8d7b4616852a011a [LEADER]: Connected to new peer: Peer: permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:52:10.446224 12651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:10.443639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:59011 (local address 127.8.154.1:45991)
0809 08:52:10.443702 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:10.443705 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:10.443714 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:52:10.443738 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:10.443739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:10.443749 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:10.443819 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:52:10.444036 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:10.444575 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:10.445386 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:10.445711 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:10.445778 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:10.445938 (+ 160us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:52:10.445975 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:52:10.446057 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:10.446128 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 08:52:10.446151 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 08:52:10.446265 12528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:10.443456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:45991 (local address 127.8.154.3:59011)
0809 08:52:10.443492 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:10.443503 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:10.443547 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:10.443843 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:10.443844 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:10.443846 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 08:52:10.443889 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:10.443890 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:10.444682 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:10.444683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:10.445284 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:10.445286 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:10.445781 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:10.445782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:10.445845 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:10.445849 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 08:52:10.446063 (+ 214us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 08:52:10.446065 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 08:52:10.446124 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 08:52:10.446183 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15}
W20260809 08:52:10.446548 12610 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 08:52:11.971480 11696 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48414:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:52:11.971580 11696 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48414:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:52:11.972486 11696 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 394558cf3144481bbebd8079b686db41: Sending DeleteTablet for 3 replicas of tablet 78374e55424442b99800c120ca55f2b2
I20260809 08:52:11.972747 11943 tablet_service.cc:1514] Processing DeleteTablet for tablet 78374e55424442b99800c120ca55f2b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:11 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:54252
I20260809 08:52:11.972780 12391 tablet_service.cc:1514] Processing DeleteTablet for tablet 78374e55424442b99800c120ca55f2b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:11 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:33100
I20260809 08:52:11.972937 12162 tablet_service.cc:1514] Processing DeleteTablet for tablet 78374e55424442b99800c120ca55f2b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:11 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:52658
I20260809 08:52:11.972978 12710 tablet_replica.cc:330] T 78374e55424442b99800c120ca55f2b2 P 30e8e1019603485185b0556dda0d8d37: stopping tablet replica
I20260809 08:52:11.973052 12710 raft_consensus.cc:2229] T 78374e55424442b99800c120ca55f2b2 P 30e8e1019603485185b0556dda0d8d37 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:52:11.973085 12710 raft_consensus.cc:2258] T 78374e55424442b99800c120ca55f2b2 P 30e8e1019603485185b0556dda0d8d37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:52:11.973152 12709 tablet_replica.cc:330] T 78374e55424442b99800c120ca55f2b2 P 3f0c22abd96e44ed8d7b4616852a011a: stopping tablet replica
I20260809 08:52:11.973210 12709 raft_consensus.cc:2229] T 78374e55424442b99800c120ca55f2b2 P 3f0c22abd96e44ed8d7b4616852a011a [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:52:11.973296 12709 raft_consensus.cc:2258] T 78374e55424442b99800c120ca55f2b2 P 3f0c22abd96e44ed8d7b4616852a011a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:52:11.973546 12709 ts_tablet_manager.cc:1905] T 78374e55424442b99800c120ca55f2b2 P 3f0c22abd96e44ed8d7b4616852a011a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:52:11.975888 12709 ts_tablet_manager.cc:1918] T 78374e55424442b99800c120ca55f2b2 P 3f0c22abd96e44ed8d7b4616852a011a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 08:52:11.976027 12709 log.cc:1196] T 78374e55424442b99800c120ca55f2b2 P 3f0c22abd96e44ed8d7b4616852a011a: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-0/wal/wals/78374e55424442b99800c120ca55f2b2
I20260809 08:52:11.976399 12710 ts_tablet_manager.cc:1905] T 78374e55424442b99800c120ca55f2b2 P 30e8e1019603485185b0556dda0d8d37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:52:11.976444 12709 ts_tablet_manager.cc:1939] T 78374e55424442b99800c120ca55f2b2 P 3f0c22abd96e44ed8d7b4616852a011a: Deleting consensus metadata
I20260809 08:52:11.976764 11678 catalog_manager.cc:4841] TS 3f0c22abd96e44ed8d7b4616852a011a (127.8.154.1:45991): tablet 78374e55424442b99800c120ca55f2b2 (table test-table [id=d2f9dbdcfac7499db3e873593225b17f]) successfully deleted
I20260809 08:52:11.977792 12710 ts_tablet_manager.cc:1918] T 78374e55424442b99800c120ca55f2b2 P 30e8e1019603485185b0556dda0d8d37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 08:52:11.977874 12710 log.cc:1196] T 78374e55424442b99800c120ca55f2b2 P 30e8e1019603485185b0556dda0d8d37: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-2/wal/wals/78374e55424442b99800c120ca55f2b2
I20260809 08:52:11.978094 12710 ts_tablet_manager.cc:1939] T 78374e55424442b99800c120ca55f2b2 P 30e8e1019603485185b0556dda0d8d37: Deleting consensus metadata
I20260809 08:52:11.978417 11679 catalog_manager.cc:4841] TS 30e8e1019603485185b0556dda0d8d37 (127.8.154.3:43665): tablet 78374e55424442b99800c120ca55f2b2 (table test-table [id=d2f9dbdcfac7499db3e873593225b17f]) successfully deleted
I20260809 08:52:11.980167 12711 tablet_replica.cc:330] T 78374e55424442b99800c120ca55f2b2 P 106032240a5c4c63bc2816062da95e6f: stopping tablet replica
I20260809 08:52:11.980225 12711 raft_consensus.cc:2229] T 78374e55424442b99800c120ca55f2b2 P 106032240a5c4c63bc2816062da95e6f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:52:11.980286 12711 raft_consensus.cc:2258] T 78374e55424442b99800c120ca55f2b2 P 106032240a5c4c63bc2816062da95e6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:52:11.980536 12711 ts_tablet_manager.cc:1905] T 78374e55424442b99800c120ca55f2b2 P 106032240a5c4c63bc2816062da95e6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:52:11.982482 12711 ts_tablet_manager.cc:1918] T 78374e55424442b99800c120ca55f2b2 P 106032240a5c4c63bc2816062da95e6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 08:52:11.982535 12711 log.cc:1196] T 78374e55424442b99800c120ca55f2b2 P 106032240a5c4c63bc2816062da95e6f: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-1/wal/wals/78374e55424442b99800c120ca55f2b2
I20260809 08:52:11.982749 12711 ts_tablet_manager.cc:1939] T 78374e55424442b99800c120ca55f2b2 P 106032240a5c4c63bc2816062da95e6f: Deleting consensus metadata
I20260809 08:52:11.983088 11679 catalog_manager.cc:4841] TS 106032240a5c4c63bc2816062da95e6f (127.8.154.2:37729): tablet 78374e55424442b99800c120ca55f2b2 (table test-table [id=d2f9dbdcfac7499db3e873593225b17f]) successfully deleted
Aug 09 08:52:11 dist-test-slave-t3q3 krb5kdc[11636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265531, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 08:52:11 dist-test-slave-t3q3 krb5kdc[11636](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265531, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:12.003051 12724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:11.992754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48428 (local address 127.8.154.62:41637)
0809 08:52:11.992907 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:11.992909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:11.992919 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:52:11.992998 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:11.992999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:11.993009 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:11.993078 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:11.993638 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:11.994174 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:11.994895 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:11.995104 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:11.995157 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:12.000750 (+ 5593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:12.000751 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:12.000751 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:12.000758 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:12.002067 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:12.002425 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:12.002426 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:12.002427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:12.002492 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:12.002655 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:12.002655 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:12.002655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:12.002756 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:12.002828 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:12.002912 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 08:52:12.002977 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":97,"threads_started":1}
I20260809 08:52:12.004557 11696 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48428:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:52:12.004716 11696 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:52:12.006114 12162 tablet_service.cc:1467] Processing CreateTablet for tablet f003ccff87bc4f9798994ceb74455c68 (DEFAULT_TABLE table=test-table [id=44a5656fb894410aaa524e887510aec8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:12.006160 12391 tablet_service.cc:1467] Processing CreateTablet for tablet f003ccff87bc4f9798994ceb74455c68 (DEFAULT_TABLE table=test-table [id=44a5656fb894410aaa524e887510aec8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:12.006170 11943 tablet_service.cc:1467] Processing CreateTablet for tablet f003ccff87bc4f9798994ceb74455c68 (DEFAULT_TABLE table=test-table [id=44a5656fb894410aaa524e887510aec8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:12.006266 12162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f003ccff87bc4f9798994ceb74455c68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:12.006282 12391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f003ccff87bc4f9798994ceb74455c68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:12.006335 11943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f003ccff87bc4f9798994ceb74455c68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:12.007462 12725 tablet_bootstrap.cc:492] T f003ccff87bc4f9798994ceb74455c68 P 3f0c22abd96e44ed8d7b4616852a011a: Bootstrap starting.
I20260809 08:52:12.007802 12730 tablet_bootstrap.cc:492] T f003ccff87bc4f9798994ceb74455c68 P 30e8e1019603485185b0556dda0d8d37: Bootstrap starting.
I20260809 08:52:12.007884 12725 tablet_bootstrap.cc:654] T f003ccff87bc4f9798994ceb74455c68 P 3f0c22abd96e44ed8d7b4616852a011a: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:12.007992 12733 tablet_bootstrap.cc:492] T f003ccff87bc4f9798994ceb74455c68 P 106032240a5c4c63bc2816062da95e6f: Bootstrap starting.
I20260809 08:52:12.008323 12730 tablet_bootstrap.cc:654] T f003ccff87bc4f9798994ceb74455c68 P 30e8e1019603485185b0556dda0d8d37: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:12.008455 12733 tablet_bootstrap.cc:654] T f003ccff87bc4f9798994ceb74455c68 P 106032240a5c4c63bc2816062da95e6f: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:12.008728 12725 tablet_bootstrap.cc:492] T f003ccff87bc4f9798994ceb74455c68 P 3f0c22abd96e44ed8d7b4616852a011a: No bootstrap required, opened a new log
I20260809 08:52:12.008797 12725 ts_tablet_manager.cc:1397] T f003ccff87bc4f9798994ceb74455c68 P 3f0c22abd96e44ed8d7b4616852a011a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 08:52:12.008989 12725 raft_consensus.cc:348] T f003ccff87bc4f9798994ceb74455c68 P 3f0c22abd96e44ed8d7b4616852a011a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } }
I20260809 08:52:12.009047 12730 tablet_bootstrap.cc:492] T f003ccff87bc4f9798994ceb74455c68 P 30e8e1019603485185b0556dda0d8d37: No bootstrap required, opened a new log
I20260809 08:52:12.009084 12725 raft_consensus.cc:374] T f003ccff87bc4f9798994ceb74455c68 P 3f0c22abd96e44ed8d7b4616852a011a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:12.009097 12730 ts_tablet_manager.cc:1397] T f003ccff87bc4f9798994ceb74455c68 P 30e8e1019603485185b0556dda0d8d37: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:12.009110 12725 raft_consensus.cc:729] T f003ccff87bc4f9798994ceb74455c68 P 3f0c22abd96e44ed8d7b4616852a011a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f0c22abd96e44ed8d7b4616852a011a, State: Initialized, Role: FOLLOWER
I20260809 08:52:12.009194 12733 tablet_bootstrap.cc:492] T f003ccff87bc4f9798994ceb74455c68 P 106032240a5c4c63bc2816062da95e6f: No bootstrap required, opened a new log
I20260809 08:52:12.009193 12725 consensus_queue.cc:260] T f003ccff87bc4f9798994ceb74455c68 P 3f0c22abd96e44ed8d7b4616852a011a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } }
I20260809 08:52:12.009253 12733 ts_tablet_manager.cc:1397] T f003ccff87bc4f9798994ceb74455c68 P 106032240a5c4c63bc2816062da95e6f: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:12.009239 12730 raft_consensus.cc:348] T f003ccff87bc4f9798994ceb74455c68 P 30e8e1019603485185b0556dda0d8d37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } }
I20260809 08:52:12.009310 12730 raft_consensus.cc:374] T f003ccff87bc4f9798994ceb74455c68 P 30e8e1019603485185b0556dda0d8d37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:12.009330 12730 raft_consensus.cc:729] T f003ccff87bc4f9798994ceb74455c68 P 30e8e1019603485185b0556dda0d8d37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30e8e1019603485185b0556dda0d8d37, State: Initialized, Role: FOLLOWER
I20260809 08:52:12.009342 12725 ts_tablet_manager.cc:1428] T f003ccff87bc4f9798994ceb74455c68 P 3f0c22abd96e44ed8d7b4616852a011a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:12.009393 12730 consensus_queue.cc:260] T f003ccff87bc4f9798994ceb74455c68 P 30e8e1019603485185b0556dda0d8d37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } }
I20260809 08:52:12.009409 12733 raft_consensus.cc:348] T f003ccff87bc4f9798994ceb74455c68 P 106032240a5c4c63bc2816062da95e6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } }
I20260809 08:52:12.009486 12733 raft_consensus.cc:374] T f003ccff87bc4f9798994ceb74455c68 P 106032240a5c4c63bc2816062da95e6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:12.009490 12730 ts_tablet_manager.cc:1428] T f003ccff87bc4f9798994ceb74455c68 P 30e8e1019603485185b0556dda0d8d37: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:12.009508 12733 raft_consensus.cc:729] T f003ccff87bc4f9798994ceb74455c68 P 106032240a5c4c63bc2816062da95e6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 106032240a5c4c63bc2816062da95e6f, State: Initialized, Role: FOLLOWER
I20260809 08:52:12.009563 12733 consensus_queue.cc:260] T f003ccff87bc4f9798994ceb74455c68 P 106032240a5c4c63bc2816062da95e6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } }
I20260809 08:52:12.009760 12733 ts_tablet_manager.cc:1428] T f003ccff87bc4f9798994ceb74455c68 P 106032240a5c4c63bc2816062da95e6f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:12.055887 12740 raft_consensus.cc:482] T f003ccff87bc4f9798994ceb74455c68 P 106032240a5c4c63bc2816062da95e6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:52:12.056107 12740 raft_consensus.cc:504] T f003ccff87bc4f9798994ceb74455c68 P 106032240a5c4c63bc2816062da95e6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } }
I20260809 08:52:12.056514 12740 leader_election.cc:288] T f003ccff87bc4f9798994ceb74455c68 P 106032240a5c4c63bc2816062da95e6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f0c22abd96e44ed8d7b4616852a011a (127.8.154.1:45991), 30e8e1019603485185b0556dda0d8d37 (127.8.154.3:43665)
I20260809 08:52:12.059628 12748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:12.056788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:35571 (local address 127.8.154.3:43665)
0809 08:52:12.057028 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:12.057031 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:12.057039 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:52:12.057072 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:12.057072 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:12.057081 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:12.057185 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:12.057367 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:12.057849 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:12.058873 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:12.059256 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:12.059340 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:12.059354 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:12.059510 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 08:52:12.059547 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":179,"threads_started":1}
I20260809 08:52:12.059653 12747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:12.056745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:40957 (local address 127.8.154.1:45991)
0809 08:52:12.056914 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:12.056917 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:12.057022 (+ 105us) server_negotiation.cc:407] Connection header received
0809 08:52:12.057068 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:12.057069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:12.057083 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:12.057176 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:12.057349 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:12.057848 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:12.058871 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:12.059258 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:12.059354 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:12.059366 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:12.059536 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 08:52:12.059576 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":89,"threads_started":1}
I20260809 08:52:12.059708 12745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:12.056721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:43665 (local address 127.8.154.2:35571)
0809 08:52:12.056908 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:12.056919 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:12.056976 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:12.057195 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:12.057196 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:12.057200 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:12.057285 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:12.057286 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:12.057949 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:12.057950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:12.058778 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:12.058781 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:12.059342 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:12.059343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:12.059423 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:12.059427 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:52:12.059490 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 08:52:12.059536 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":52,"thread_start_us":110,"threads_started":1}
I20260809 08:52:12.060025 12741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:12.056683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:45991 (local address 127.8.154.2:40957)
0809 08:52:12.056889 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:12.056903 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:12.056994 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:12.057177 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:12.057178 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:12.057181 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:12.057268 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:12.057270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:12.057941 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:12.057942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:12.058770 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:12.058772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:12.059356 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:12.059357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:12.059434 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:12.059438 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:52:12.059522 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 08:52:12.059567 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":2,"thread_start_us":123,"threads_started":1}
I20260809 08:52:12.060105 12435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f003ccff87bc4f9798994ceb74455c68" candidate_uuid: "106032240a5c4c63bc2816062da95e6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30e8e1019603485185b0556dda0d8d37" is_pre_election: true
I20260809 08:52:12.060226 12435 raft_consensus.cc:2454] T f003ccff87bc4f9798994ceb74455c68 P 30e8e1019603485185b0556dda0d8d37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 106032240a5c4c63bc2816062da95e6f in term 0.
I20260809 08:52:12.060331 11971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f003ccff87bc4f9798994ceb74455c68" candidate_uuid: "106032240a5c4c63bc2816062da95e6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f0c22abd96e44ed8d7b4616852a011a" is_pre_election: true
I20260809 08:52:12.060423 11971 raft_consensus.cc:2454] T f003ccff87bc4f9798994ceb74455c68 P 3f0c22abd96e44ed8d7b4616852a011a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 106032240a5c4c63bc2816062da95e6f in term 0.
I20260809 08:52:12.060472 12090 leader_election.cc:302] T f003ccff87bc4f9798994ceb74455c68 P 106032240a5c4c63bc2816062da95e6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 106032240a5c4c63bc2816062da95e6f, 30e8e1019603485185b0556dda0d8d37; no voters:
I20260809 08:52:12.060665 12740 raft_consensus.cc:2781] T f003ccff87bc4f9798994ceb74455c68 P 106032240a5c4c63bc2816062da95e6f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:52:12.060698 12740 raft_consensus.cc:482] T f003ccff87bc4f9798994ceb74455c68 P 106032240a5c4c63bc2816062da95e6f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:52:12.060714 12740 raft_consensus.cc:3037] T f003ccff87bc4f9798994ceb74455c68 P 106032240a5c4c63bc2816062da95e6f [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:12.061452 12740 raft_consensus.cc:504] T f003ccff87bc4f9798994ceb74455c68 P 106032240a5c4c63bc2816062da95e6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } }
I20260809 08:52:12.061672 12740 leader_election.cc:288] T f003ccff87bc4f9798994ceb74455c68 P 106032240a5c4c63bc2816062da95e6f [CANDIDATE]: Term 1 election: Requested vote from peers 3f0c22abd96e44ed8d7b4616852a011a (127.8.154.1:45991), 30e8e1019603485185b0556dda0d8d37 (127.8.154.3:43665)
I20260809 08:52:12.061738 12435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f003ccff87bc4f9798994ceb74455c68" candidate_uuid: "106032240a5c4c63bc2816062da95e6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30e8e1019603485185b0556dda0d8d37"
I20260809 08:52:12.061723 11971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f003ccff87bc4f9798994ceb74455c68" candidate_uuid: "106032240a5c4c63bc2816062da95e6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f0c22abd96e44ed8d7b4616852a011a"
I20260809 08:52:12.061806 11971 raft_consensus.cc:3037] T f003ccff87bc4f9798994ceb74455c68 P 3f0c22abd96e44ed8d7b4616852a011a [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:12.061817 12435 raft_consensus.cc:3037] T f003ccff87bc4f9798994ceb74455c68 P 30e8e1019603485185b0556dda0d8d37 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:12.062572 12435 raft_consensus.cc:2454] T f003ccff87bc4f9798994ceb74455c68 P 30e8e1019603485185b0556dda0d8d37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 106032240a5c4c63bc2816062da95e6f in term 1.
I20260809 08:52:12.062572 11971 raft_consensus.cc:2454] T f003ccff87bc4f9798994ceb74455c68 P 3f0c22abd96e44ed8d7b4616852a011a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 106032240a5c4c63bc2816062da95e6f in term 1.
I20260809 08:52:12.062743 12090 leader_election.cc:302] T f003ccff87bc4f9798994ceb74455c68 P 106032240a5c4c63bc2816062da95e6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 106032240a5c4c63bc2816062da95e6f, 30e8e1019603485185b0556dda0d8d37; no voters:
I20260809 08:52:12.062876 12740 raft_consensus.cc:2781] T f003ccff87bc4f9798994ceb74455c68 P 106032240a5c4c63bc2816062da95e6f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:52:12.063057 12740 raft_consensus.cc:686] T f003ccff87bc4f9798994ceb74455c68 P 106032240a5c4c63bc2816062da95e6f [term 1 LEADER]: Becoming Leader. State: Replica: 106032240a5c4c63bc2816062da95e6f, State: Running, Role: LEADER
I20260809 08:52:12.063154 12740 consensus_queue.cc:237] T f003ccff87bc4f9798994ceb74455c68 P 106032240a5c4c63bc2816062da95e6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } }
I20260809 08:52:12.063923 11697 catalog_manager.cc:5495] T f003ccff87bc4f9798994ceb74455c68 P 106032240a5c4c63bc2816062da95e6f reported cstate change: term changed from 0 to 1, leader changed from <none> to 106032240a5c4c63bc2816062da95e6f (127.8.154.2). New cstate: current_term: 1 leader_uuid: "106032240a5c4c63bc2816062da95e6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } health_report { overall_health: UNKNOWN } } }
I20260809 08:52:12.086645 12754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:12.083969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52660 (local address 127.8.154.2:37729)
0809 08:52:12.084177 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:12.084179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:12.084185 (+ 6us) server_negotiation.cc:407] Connection header received
0809 08:52:12.084219 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:12.084219 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:12.084239 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:12.084337 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:52:12.084457 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:12.084963 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:12.085769 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:12.085995 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:12.086072 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:12.086253 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:52:12.086372 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:52:12.086458 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:12.086520 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 08:52:12.086542 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":137,"threads_started":1}
I20260809 08:52:12.091902 12748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:12.089432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:51281 (local address 127.8.154.3:43665)
0809 08:52:12.089503 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:12.089505 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:12.089645 (+ 140us) server_negotiation.cc:407] Connection header received
0809 08:52:12.089710 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:12.089711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:12.089725 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:12.089792 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:52:12.089939 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:12.090392 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:12.091169 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:12.091360 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:12.091436 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:12.091644 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:52:12.091675 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:52:12.091729 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:12.091807 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 08:52:12.091828 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 08:52:12.091929 12758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:12.089292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:43665 (local address 127.8.154.2:51281)
0809 08:52:12.089572 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:12.089581 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:12.089654 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:12.089796 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:12.089798 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:12.089799 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 08:52:12.089869 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:12.089870 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:12.090472 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:12.090473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:12.091073 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:12.091075 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:12.091437 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:12.091437 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:12.091521 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:12.091540 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 08:52:12.091760 (+ 220us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 08:52:12.091762 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 08:52:12.091804 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 08:52:12.091846 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":98,"threads_started":1}
I20260809 08:52:12.094365 11971 raft_consensus.cc:1261] T f003ccff87bc4f9798994ceb74455c68 P 3f0c22abd96e44ed8d7b4616852a011a [term 1 FOLLOWER]: Refusing update from remote peer 106032240a5c4c63bc2816062da95e6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:52:12.094451 12435 raft_consensus.cc:1261] T f003ccff87bc4f9798994ceb74455c68 P 30e8e1019603485185b0556dda0d8d37 [term 1 FOLLOWER]: Refusing update from remote peer 106032240a5c4c63bc2816062da95e6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:52:12.094653 12749 consensus_queue.cc:1035] T f003ccff87bc4f9798994ceb74455c68 P 106032240a5c4c63bc2816062da95e6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:52:12.094775 12740 consensus_queue.cc:1035] T f003ccff87bc4f9798994ceb74455c68 P 106032240a5c4c63bc2816062da95e6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:52:12.102237 12754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:12.099823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:45409 (local address 127.8.154.2:37729)
0809 08:52:12.099866 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:12.099868 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:12.099993 (+ 125us) server_negotiation.cc:407] Connection header received
0809 08:52:12.100039 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:12.100040 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:12.100047 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:12.100141 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:52:12.100275 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:12.100726 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:12.101573 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:12.101861 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:12.101924 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:12.102082 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:52:12.102106 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:52:12.102141 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:12.102177 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 08:52:12.102192 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:52:12.102270 12764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:12.099750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:37729 (local address 127.8.154.3:45409)
0809 08:52:12.099925 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:12.099935 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:12.099994 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:12.100155 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:12.100156 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:12.100161 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 08:52:12.100201 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:12.100202 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:12.100790 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:12.100790 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:12.101480 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:12.101484 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:12.101936 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:12.101938 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:12.101993 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:12.101997 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 08:52:12.102145 (+ 148us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 08:52:12.102146 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 08:52:12.102177 (+ 31us) client_negotiation.cc:240] Negotiation successful
0809 08:52:12.102214 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":89,"threads_started":1}
I20260809 08:52:12.109016 11698 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48428:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:52:12.109108 11698 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48428:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:52:12.109723 11698 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 394558cf3144481bbebd8079b686db41: Sending DeleteTablet for 3 replicas of tablet f003ccff87bc4f9798994ceb74455c68
I20260809 08:52:12.109966 11943 tablet_service.cc:1514] Processing DeleteTablet for tablet f003ccff87bc4f9798994ceb74455c68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:12 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:54252
I20260809 08:52:12.110009 12391 tablet_service.cc:1514] Processing DeleteTablet for tablet f003ccff87bc4f9798994ceb74455c68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:12 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:33100
I20260809 08:52:12.110015 12162 tablet_service.cc:1514] Processing DeleteTablet for tablet f003ccff87bc4f9798994ceb74455c68 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:12 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:52658
I20260809 08:52:12.110100 12711 tablet_replica.cc:330] T f003ccff87bc4f9798994ceb74455c68 P 106032240a5c4c63bc2816062da95e6f: stopping tablet replica
I20260809 08:52:12.110108 12709 tablet_replica.cc:330] T f003ccff87bc4f9798994ceb74455c68 P 3f0c22abd96e44ed8d7b4616852a011a: stopping tablet replica
I20260809 08:52:12.110152 12711 raft_consensus.cc:2229] T f003ccff87bc4f9798994ceb74455c68 P 106032240a5c4c63bc2816062da95e6f [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:52:12.110163 12709 raft_consensus.cc:2229] T f003ccff87bc4f9798994ceb74455c68 P 3f0c22abd96e44ed8d7b4616852a011a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:52:12.110200 12709 raft_consensus.cc:2258] T f003ccff87bc4f9798994ceb74455c68 P 3f0c22abd96e44ed8d7b4616852a011a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:52:12.110258 12711 raft_consensus.cc:2258] T f003ccff87bc4f9798994ceb74455c68 P 106032240a5c4c63bc2816062da95e6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:52:12.110278 12710 tablet_replica.cc:330] T f003ccff87bc4f9798994ceb74455c68 P 30e8e1019603485185b0556dda0d8d37: stopping tablet replica
I20260809 08:52:12.110344 12710 raft_consensus.cc:2229] T f003ccff87bc4f9798994ceb74455c68 P 30e8e1019603485185b0556dda0d8d37 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:52:12.110397 12710 raft_consensus.cc:2258] T f003ccff87bc4f9798994ceb74455c68 P 30e8e1019603485185b0556dda0d8d37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:52:12.110605 12711 ts_tablet_manager.cc:1905] T f003ccff87bc4f9798994ceb74455c68 P 106032240a5c4c63bc2816062da95e6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:52:12.110703 12709 ts_tablet_manager.cc:1905] T f003ccff87bc4f9798994ceb74455c68 P 3f0c22abd96e44ed8d7b4616852a011a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:52:12.110795 12710 ts_tablet_manager.cc:1905] T f003ccff87bc4f9798994ceb74455c68 P 30e8e1019603485185b0556dda0d8d37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:52:12.112321 12710 ts_tablet_manager.cc:1918] T f003ccff87bc4f9798994ceb74455c68 P 30e8e1019603485185b0556dda0d8d37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:52:12.112323 12709 ts_tablet_manager.cc:1918] T f003ccff87bc4f9798994ceb74455c68 P 3f0c22abd96e44ed8d7b4616852a011a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:52:12.112376 12710 log.cc:1196] T f003ccff87bc4f9798994ceb74455c68 P 30e8e1019603485185b0556dda0d8d37: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-2/wal/wals/f003ccff87bc4f9798994ceb74455c68
I20260809 08:52:12.112380 12709 log.cc:1196] T f003ccff87bc4f9798994ceb74455c68 P 3f0c22abd96e44ed8d7b4616852a011a: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-0/wal/wals/f003ccff87bc4f9798994ceb74455c68
I20260809 08:52:12.112607 12709 ts_tablet_manager.cc:1939] T f003ccff87bc4f9798994ceb74455c68 P 3f0c22abd96e44ed8d7b4616852a011a: Deleting consensus metadata
I20260809 08:52:12.112636 12710 ts_tablet_manager.cc:1939] T f003ccff87bc4f9798994ceb74455c68 P 30e8e1019603485185b0556dda0d8d37: Deleting consensus metadata
I20260809 08:52:12.112560 11698 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48428:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:52:12.112715 11698 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:52:12.112830 12711 ts_tablet_manager.cc:1918] T f003ccff87bc4f9798994ceb74455c68 P 106032240a5c4c63bc2816062da95e6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:52:12.112890 12711 log.cc:1196] T f003ccff87bc4f9798994ceb74455c68 P 106032240a5c4c63bc2816062da95e6f: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-1/wal/wals/f003ccff87bc4f9798994ceb74455c68
I20260809 08:52:12.113216 12711 ts_tablet_manager.cc:1939] T f003ccff87bc4f9798994ceb74455c68 P 106032240a5c4c63bc2816062da95e6f: Deleting consensus metadata
I20260809 08:52:12.113516 11679 catalog_manager.cc:4841] TS 106032240a5c4c63bc2816062da95e6f (127.8.154.2:37729): tablet f003ccff87bc4f9798994ceb74455c68 (table test-table [id=44a5656fb894410aaa524e887510aec8]) successfully deleted
I20260809 08:52:12.113778 11679 catalog_manager.cc:4841] TS 30e8e1019603485185b0556dda0d8d37 (127.8.154.3:43665): tablet f003ccff87bc4f9798994ceb74455c68 (table test-table [id=44a5656fb894410aaa524e887510aec8]) successfully deleted
I20260809 08:52:12.113870 11678 catalog_manager.cc:4841] TS 3f0c22abd96e44ed8d7b4616852a011a (127.8.154.1:45991): tablet f003ccff87bc4f9798994ceb74455c68 (table test-table [id=44a5656fb894410aaa524e887510aec8]) successfully deleted
I20260809 08:52:12.114233 11943 tablet_service.cc:1467] Processing CreateTablet for tablet 78c2cd4d92674483b24bb9721abd8516 (DEFAULT_TABLE table=test-table [id=77ff1d23c65b49b5b88598127e805d32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:12.114264 12391 tablet_service.cc:1467] Processing CreateTablet for tablet 78c2cd4d92674483b24bb9721abd8516 (DEFAULT_TABLE table=test-table [id=77ff1d23c65b49b5b88598127e805d32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:12.114290 12162 tablet_service.cc:1467] Processing CreateTablet for tablet 78c2cd4d92674483b24bb9721abd8516 (DEFAULT_TABLE table=test-table [id=77ff1d23c65b49b5b88598127e805d32]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:12.114342 11943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78c2cd4d92674483b24bb9721abd8516. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:12.114372 12391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78c2cd4d92674483b24bb9721abd8516. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:12.114373 12162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 78c2cd4d92674483b24bb9721abd8516. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:12.115523 12733 tablet_bootstrap.cc:492] T 78c2cd4d92674483b24bb9721abd8516 P 106032240a5c4c63bc2816062da95e6f: Bootstrap starting.
I20260809 08:52:12.115516 12730 tablet_bootstrap.cc:492] T 78c2cd4d92674483b24bb9721abd8516 P 30e8e1019603485185b0556dda0d8d37: Bootstrap starting.
I20260809 08:52:12.115545 12725 tablet_bootstrap.cc:492] T 78c2cd4d92674483b24bb9721abd8516 P 3f0c22abd96e44ed8d7b4616852a011a: Bootstrap starting.
I20260809 08:52:12.115833 12730 tablet_bootstrap.cc:654] T 78c2cd4d92674483b24bb9721abd8516 P 30e8e1019603485185b0556dda0d8d37: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:12.115833 12725 tablet_bootstrap.cc:654] T 78c2cd4d92674483b24bb9721abd8516 P 3f0c22abd96e44ed8d7b4616852a011a: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:12.115993 12733 tablet_bootstrap.cc:654] T 78c2cd4d92674483b24bb9721abd8516 P 106032240a5c4c63bc2816062da95e6f: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:12.116525 12725 tablet_bootstrap.cc:492] T 78c2cd4d92674483b24bb9721abd8516 P 3f0c22abd96e44ed8d7b4616852a011a: No bootstrap required, opened a new log
I20260809 08:52:12.116580 12725 ts_tablet_manager.cc:1397] T 78c2cd4d92674483b24bb9721abd8516 P 3f0c22abd96e44ed8d7b4616852a011a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 08:52:12.116683 12730 tablet_bootstrap.cc:492] T 78c2cd4d92674483b24bb9721abd8516 P 30e8e1019603485185b0556dda0d8d37: No bootstrap required, opened a new log
I20260809 08:52:12.116743 12730 ts_tablet_manager.cc:1397] T 78c2cd4d92674483b24bb9721abd8516 P 30e8e1019603485185b0556dda0d8d37: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:12.116715 12725 raft_consensus.cc:348] T 78c2cd4d92674483b24bb9721abd8516 P 3f0c22abd96e44ed8d7b4616852a011a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } }
I20260809 08:52:12.116783 12725 raft_consensus.cc:374] T 78c2cd4d92674483b24bb9721abd8516 P 3f0c22abd96e44ed8d7b4616852a011a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:12.116804 12725 raft_consensus.cc:729] T 78c2cd4d92674483b24bb9721abd8516 P 3f0c22abd96e44ed8d7b4616852a011a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f0c22abd96e44ed8d7b4616852a011a, State: Initialized, Role: FOLLOWER
I20260809 08:52:12.116844 12733 tablet_bootstrap.cc:492] T 78c2cd4d92674483b24bb9721abd8516 P 106032240a5c4c63bc2816062da95e6f: No bootstrap required, opened a new log
I20260809 08:52:12.116842 12725 consensus_queue.cc:260] T 78c2cd4d92674483b24bb9721abd8516 P 3f0c22abd96e44ed8d7b4616852a011a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } }
I20260809 08:52:12.116888 12733 ts_tablet_manager.cc:1397] T 78c2cd4d92674483b24bb9721abd8516 P 106032240a5c4c63bc2816062da95e6f: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 08:52:12.116853 12730 raft_consensus.cc:348] T 78c2cd4d92674483b24bb9721abd8516 P 30e8e1019603485185b0556dda0d8d37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } }
I20260809 08:52:12.116922 12730 raft_consensus.cc:374] T 78c2cd4d92674483b24bb9721abd8516 P 30e8e1019603485185b0556dda0d8d37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:12.116927 12725 ts_tablet_manager.cc:1428] T 78c2cd4d92674483b24bb9721abd8516 P 3f0c22abd96e44ed8d7b4616852a011a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:12.116947 12730 raft_consensus.cc:729] T 78c2cd4d92674483b24bb9721abd8516 P 30e8e1019603485185b0556dda0d8d37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30e8e1019603485185b0556dda0d8d37, State: Initialized, Role: FOLLOWER
I20260809 08:52:12.116997 12730 consensus_queue.cc:260] T 78c2cd4d92674483b24bb9721abd8516 P 30e8e1019603485185b0556dda0d8d37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } }
I20260809 08:52:12.117084 12730 ts_tablet_manager.cc:1428] T 78c2cd4d92674483b24bb9721abd8516 P 30e8e1019603485185b0556dda0d8d37: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:12.117058 12733 raft_consensus.cc:348] T 78c2cd4d92674483b24bb9721abd8516 P 106032240a5c4c63bc2816062da95e6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } }
I20260809 08:52:12.117120 12733 raft_consensus.cc:374] T 78c2cd4d92674483b24bb9721abd8516 P 106032240a5c4c63bc2816062da95e6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:12.117144 12733 raft_consensus.cc:729] T 78c2cd4d92674483b24bb9721abd8516 P 106032240a5c4c63bc2816062da95e6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 106032240a5c4c63bc2816062da95e6f, State: Initialized, Role: FOLLOWER
I20260809 08:52:12.117189 12733 consensus_queue.cc:260] T 78c2cd4d92674483b24bb9721abd8516 P 106032240a5c4c63bc2816062da95e6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } }
I20260809 08:52:12.117321 12733 ts_tablet_manager.cc:1428] T 78c2cd4d92674483b24bb9721abd8516 P 106032240a5c4c63bc2816062da95e6f: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:12.208983 12740 raft_consensus.cc:482] T 78c2cd4d92674483b24bb9721abd8516 P 106032240a5c4c63bc2816062da95e6f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:52:12.209079 12740 raft_consensus.cc:504] T 78c2cd4d92674483b24bb9721abd8516 P 106032240a5c4c63bc2816062da95e6f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } }
I20260809 08:52:12.209277 12740 leader_election.cc:288] T 78c2cd4d92674483b24bb9721abd8516 P 106032240a5c4c63bc2816062da95e6f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f0c22abd96e44ed8d7b4616852a011a (127.8.154.1:45991), 30e8e1019603485185b0556dda0d8d37 (127.8.154.3:43665)
I20260809 08:52:12.209546 12435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78c2cd4d92674483b24bb9721abd8516" candidate_uuid: "106032240a5c4c63bc2816062da95e6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30e8e1019603485185b0556dda0d8d37" is_pre_election: true
I20260809 08:52:12.209545 11971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78c2cd4d92674483b24bb9721abd8516" candidate_uuid: "106032240a5c4c63bc2816062da95e6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f0c22abd96e44ed8d7b4616852a011a" is_pre_election: true
I20260809 08:52:12.209662 12435 raft_consensus.cc:2454] T 78c2cd4d92674483b24bb9721abd8516 P 30e8e1019603485185b0556dda0d8d37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 106032240a5c4c63bc2816062da95e6f in term 0.
I20260809 08:52:12.209664 11971 raft_consensus.cc:2454] T 78c2cd4d92674483b24bb9721abd8516 P 3f0c22abd96e44ed8d7b4616852a011a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 106032240a5c4c63bc2816062da95e6f in term 0.
I20260809 08:52:12.209874 12090 leader_election.cc:302] T 78c2cd4d92674483b24bb9721abd8516 P 106032240a5c4c63bc2816062da95e6f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 106032240a5c4c63bc2816062da95e6f, 30e8e1019603485185b0556dda0d8d37; no voters:
I20260809 08:52:12.209970 12740 raft_consensus.cc:2781] T 78c2cd4d92674483b24bb9721abd8516 P 106032240a5c4c63bc2816062da95e6f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:52:12.210008 12740 raft_consensus.cc:482] T 78c2cd4d92674483b24bb9721abd8516 P 106032240a5c4c63bc2816062da95e6f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:52:12.210027 12740 raft_consensus.cc:3037] T 78c2cd4d92674483b24bb9721abd8516 P 106032240a5c4c63bc2816062da95e6f [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:12.210780 12740 raft_consensus.cc:504] T 78c2cd4d92674483b24bb9721abd8516 P 106032240a5c4c63bc2816062da95e6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } }
I20260809 08:52:12.210919 12740 leader_election.cc:288] T 78c2cd4d92674483b24bb9721abd8516 P 106032240a5c4c63bc2816062da95e6f [CANDIDATE]: Term 1 election: Requested vote from peers 3f0c22abd96e44ed8d7b4616852a011a (127.8.154.1:45991), 30e8e1019603485185b0556dda0d8d37 (127.8.154.3:43665)
I20260809 08:52:12.211100 11971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78c2cd4d92674483b24bb9721abd8516" candidate_uuid: "106032240a5c4c63bc2816062da95e6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f0c22abd96e44ed8d7b4616852a011a"
I20260809 08:52:12.211196 11971 raft_consensus.cc:3037] T 78c2cd4d92674483b24bb9721abd8516 P 3f0c22abd96e44ed8d7b4616852a011a [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:12.211143 12435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "78c2cd4d92674483b24bb9721abd8516" candidate_uuid: "106032240a5c4c63bc2816062da95e6f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30e8e1019603485185b0556dda0d8d37"
I20260809 08:52:12.211262 12435 raft_consensus.cc:3037] T 78c2cd4d92674483b24bb9721abd8516 P 30e8e1019603485185b0556dda0d8d37 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:12.211889 12435 raft_consensus.cc:2454] T 78c2cd4d92674483b24bb9721abd8516 P 30e8e1019603485185b0556dda0d8d37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 106032240a5c4c63bc2816062da95e6f in term 1.
I20260809 08:52:12.212003 11971 raft_consensus.cc:2454] T 78c2cd4d92674483b24bb9721abd8516 P 3f0c22abd96e44ed8d7b4616852a011a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 106032240a5c4c63bc2816062da95e6f in term 1.
I20260809 08:52:12.212121 12090 leader_election.cc:302] T 78c2cd4d92674483b24bb9721abd8516 P 106032240a5c4c63bc2816062da95e6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 106032240a5c4c63bc2816062da95e6f, 30e8e1019603485185b0556dda0d8d37; no voters:
I20260809 08:52:12.212215 12740 raft_consensus.cc:2781] T 78c2cd4d92674483b24bb9721abd8516 P 106032240a5c4c63bc2816062da95e6f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:52:12.212286 12740 raft_consensus.cc:686] T 78c2cd4d92674483b24bb9721abd8516 P 106032240a5c4c63bc2816062da95e6f [term 1 LEADER]: Becoming Leader. State: Replica: 106032240a5c4c63bc2816062da95e6f, State: Running, Role: LEADER
I20260809 08:52:12.212347 12740 consensus_queue.cc:237] T 78c2cd4d92674483b24bb9721abd8516 P 106032240a5c4c63bc2816062da95e6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } }
I20260809 08:52:12.213039 11697 catalog_manager.cc:5495] T 78c2cd4d92674483b24bb9721abd8516 P 106032240a5c4c63bc2816062da95e6f reported cstate change: term changed from 0 to 1, leader changed from <none> to 106032240a5c4c63bc2816062da95e6f (127.8.154.2). New cstate: current_term: 1 leader_uuid: "106032240a5c4c63bc2816062da95e6f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "106032240a5c4c63bc2816062da95e6f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 } health_report { overall_health: UNKNOWN } } }
I20260809 08:52:12.238555 12435 raft_consensus.cc:1261] T 78c2cd4d92674483b24bb9721abd8516 P 30e8e1019603485185b0556dda0d8d37 [term 1 FOLLOWER]: Refusing update from remote peer 106032240a5c4c63bc2816062da95e6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:52:12.238592 11971 raft_consensus.cc:1261] T 78c2cd4d92674483b24bb9721abd8516 P 3f0c22abd96e44ed8d7b4616852a011a [term 1 FOLLOWER]: Refusing update from remote peer 106032240a5c4c63bc2816062da95e6f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:52:12.238798 12740 consensus_queue.cc:1035] T 78c2cd4d92674483b24bb9721abd8516 P 106032240a5c4c63bc2816062da95e6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "30e8e1019603485185b0556dda0d8d37" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 43665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:52:12.238948 12749 consensus_queue.cc:1035] T 78c2cd4d92674483b24bb9721abd8516 P 106032240a5c4c63bc2816062da95e6f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f0c22abd96e44ed8d7b4616852a011a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45991 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 08:52:12.243414 12675 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 08:52:13.764816 11698 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48428:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:52:13.764950 11698 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48428:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:52:13.765838 11698 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 394558cf3144481bbebd8079b686db41: Sending DeleteTablet for 3 replicas of tablet 78c2cd4d92674483b24bb9721abd8516
I20260809 08:52:13.766095 11943 tablet_service.cc:1514] Processing DeleteTablet for tablet 78c2cd4d92674483b24bb9721abd8516 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:13 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:54252
I20260809 08:52:13.766304 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 11825
I20260809 08:52:13.766311 12162 tablet_service.cc:1514] Processing DeleteTablet for tablet 78c2cd4d92674483b24bb9721abd8516 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:13 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:52658
I20260809 08:52:13.766479 12390 tablet_service.cc:1514] Processing DeleteTablet for tablet 78c2cd4d92674483b24bb9721abd8516 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:13 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:33100
I20260809 08:52:13.776263 12810 tablet_replica.cc:330] T 78c2cd4d92674483b24bb9721abd8516 P 30e8e1019603485185b0556dda0d8d37: stopping tablet replica
I20260809 08:52:13.776263 12809 tablet_replica.cc:330] T 78c2cd4d92674483b24bb9721abd8516 P 106032240a5c4c63bc2816062da95e6f: stopping tablet replica
I20260809 08:52:13.776366 12810 raft_consensus.cc:2229] T 78c2cd4d92674483b24bb9721abd8516 P 30e8e1019603485185b0556dda0d8d37 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:52:13.776367 12809 raft_consensus.cc:2229] T 78c2cd4d92674483b24bb9721abd8516 P 106032240a5c4c63bc2816062da95e6f [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:52:13.776417 12810 raft_consensus.cc:2258] T 78c2cd4d92674483b24bb9721abd8516 P 30e8e1019603485185b0556dda0d8d37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:52:13.776448 12809 raft_consensus.cc:2258] T 78c2cd4d92674483b24bb9721abd8516 P 106032240a5c4c63bc2816062da95e6f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:52:13.776734 12810 ts_tablet_manager.cc:1905] T 78c2cd4d92674483b24bb9721abd8516 P 30e8e1019603485185b0556dda0d8d37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:52:13.776734 12809 ts_tablet_manager.cc:1905] T 78c2cd4d92674483b24bb9721abd8516 P 106032240a5c4c63bc2816062da95e6f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:52:13.778626 12810 ts_tablet_manager.cc:1918] T 78c2cd4d92674483b24bb9721abd8516 P 30e8e1019603485185b0556dda0d8d37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 08:52:13.778640 12809 ts_tablet_manager.cc:1918] T 78c2cd4d92674483b24bb9721abd8516 P 106032240a5c4c63bc2816062da95e6f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 08:52:13.778703 12809 log.cc:1196] T 78c2cd4d92674483b24bb9721abd8516 P 106032240a5c4c63bc2816062da95e6f: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-1/wal/wals/78c2cd4d92674483b24bb9721abd8516
I20260809 08:52:13.778721 12810 log.cc:1196] T 78c2cd4d92674483b24bb9721abd8516 P 30e8e1019603485185b0556dda0d8d37: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265519942966-8808-0/minicluster-data/ts-2/wal/wals/78c2cd4d92674483b24bb9721abd8516
I20260809 08:52:13.778957 12810 ts_tablet_manager.cc:1939] T 78c2cd4d92674483b24bb9721abd8516 P 30e8e1019603485185b0556dda0d8d37: Deleting consensus metadata
I20260809 08:52:13.779137 12809 ts_tablet_manager.cc:1939] T 78c2cd4d92674483b24bb9721abd8516 P 106032240a5c4c63bc2816062da95e6f: Deleting consensus metadata
I20260809 08:52:13.779417 11679 catalog_manager.cc:4841] TS 30e8e1019603485185b0556dda0d8d37 (127.8.154.3:43665): tablet 78c2cd4d92674483b24bb9721abd8516 (table test-table [id=77ff1d23c65b49b5b88598127e805d32]) successfully deleted
I20260809 08:52:13.779533 11679 catalog_manager.cc:4841] TS 106032240a5c4c63bc2816062da95e6f (127.8.154.2:37729): tablet 78c2cd4d92674483b24bb9721abd8516 (table test-table [id=77ff1d23c65b49b5b88598127e805d32]) successfully deleted
W20260809 08:52:13.785157 11678 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.154.1:45991 (error 108)
I20260809 08:52:13.785248 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 12056
I20260809 08:52:13.797060 12812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:13.795303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:45991 (local address 127.0.0.1:54274)
0809 08:52:13.796920 (+ 1617us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:13.796994 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.1:45991: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1582,"thread_start_us":505,"threads_started":1}
W20260809 08:52:13.799301 11678 catalog_manager.cc:4584] TS 3f0c22abd96e44ed8d7b4616852a011a (127.8.154.1:45991): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 78c2cd4d92674483b24bb9721abd8516: Network error: Client connection negotiation failed: client connection to 127.8.154.1:45991: connect: Connection refused (error 111)
I20260809 08:52:13.804674 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 12284
I20260809 08:52:13.816759 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 11654
2026-08-09T08:52:13Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6330 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:52:13 dist-test-slave-t3q3 krb5kdc[12817](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:52:13 dist-test-slave-t3q3 krb5kdc[12817](info): set up 2 sockets
Aug 09 08:52:13 dist-test-slave-t3q3 krb5kdc[12817](info): commencing operation
krb5kdc: starting...
W20260809 08:52:15.404316 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.558s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:52:15 dist-test-slave-t3q3 krb5kdc[12817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:52:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:52:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
I20260809 08:52:15.598223 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:36523
--webserver_interface=127.8.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:40909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:36523
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:15.751704 12847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:15.752192 12847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:15.752349 12847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:15.754387 12847 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:52:15.754503 12847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:15.754596 12847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:15.754719 12847 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:52:15.754832 12847 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:52:15.758108 12847 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:40909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:36523
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:36523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:15.758728 12847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:15.759243 12847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:15.769135 12860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:15.769173 12858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:15.769233 12847 server_base.cc:1034] running on GCE node
W20260809 08:52:15.769173 12863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:15.770231 12847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:15.770777 12847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:15.772037 12847 hybrid_clock.cc:648] HybridClock initialized: now 1786265535772000 us; error 85 us; skew 500 ppm
Aug 09 08:52:15 dist-test-slave-t3q3 krb5kdc[12817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265535, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:15.777211 12847 init.cc:377] Logged in from keytab as kudu/127.8.154.62@KRBTEST.COM (short username kudu)
I20260809 08:52:15.778214 12847 webserver.cc:460] Webserver started at http://127.8.154.62:36165/ using document root <none> and password file <none>
I20260809 08:52:15.778633 12847 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:15.778805 12847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:15.779119 12847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:15.781157 12847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "a8a338b3fc724893a8ed3b087497276c"
format_stamp: "Formatted at 2026-08-09 08:52:15 on dist-test-slave-t3q3"
server_key: "bacf76d07f74399131500501c8941950"
server_key_iv: "e957f1cebbe63f82d1c67a1da9081c08"
server_key_version: "encryptionkey@0"
I20260809 08:52:15.781845 12847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "a8a338b3fc724893a8ed3b087497276c"
format_stamp: "Formatted at 2026-08-09 08:52:15 on dist-test-slave-t3q3"
server_key: "bacf76d07f74399131500501c8941950"
server_key_iv: "e957f1cebbe63f82d1c67a1da9081c08"
server_key_version: "encryptionkey@0"
I20260809 08:52:15.784792 12847 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260809 08:52:15.786283 12872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:15.786654 12847 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:15.786844 12847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "a8a338b3fc724893a8ed3b087497276c"
format_stamp: "Formatted at 2026-08-09 08:52:15 on dist-test-slave-t3q3"
server_key: "bacf76d07f74399131500501c8941950"
server_key_iv: "e957f1cebbe63f82d1c67a1da9081c08"
server_key_version: "encryptionkey@0"
I20260809 08:52:15.787002 12847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:15.810729 12847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:15.815006 12847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:15.815299 12847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:15.823321 12847 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:36523
I20260809 08:52:15.823416 12962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:36523 every 8 connection(s)
I20260809 08:52:15.824282 12847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:52:15.825832 12963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:15.828841 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 12847
I20260809 08:52:15.829017 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:52:15.829404 8808 external_mini_cluster.cc:1428] Setting key 90e55cfa555e13bb1b7a2f2be2be337a
I20260809 08:52:15.831115 12963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8a338b3fc724893a8ed3b087497276c: Bootstrap starting.
I20260809 08:52:15.832504 12963 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8a338b3fc724893a8ed3b087497276c: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:15.833184 12963 log.cc:824] T 00000000000000000000000000000000 P a8a338b3fc724893a8ed3b087497276c: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:15.836635 12963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8a338b3fc724893a8ed3b087497276c: No bootstrap required, opened a new log
I20260809 08:52:15.841208 12963 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8a338b3fc724893a8ed3b087497276c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8a338b3fc724893a8ed3b087497276c" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 36523 } }
I20260809 08:52:15.841506 12963 raft_consensus.cc:374] T 00000000000000000000000000000000 P a8a338b3fc724893a8ed3b087497276c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:15.841617 12963 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8a338b3fc724893a8ed3b087497276c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8a338b3fc724893a8ed3b087497276c, State: Initialized, Role: FOLLOWER
I20260809 08:52:15.841995 12963 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8a338b3fc724893a8ed3b087497276c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8a338b3fc724893a8ed3b087497276c" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 36523 } }
I20260809 08:52:15.842185 12963 raft_consensus.cc:388] T 00000000000000000000000000000000 P a8a338b3fc724893a8ed3b087497276c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:52:15.842254 12963 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8a338b3fc724893a8ed3b087497276c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:52:15.842348 12963 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8a338b3fc724893a8ed3b087497276c [term 0 FOLLOWER]: Advancing to term 1
Aug 09 08:52:15 dist-test-slave-t3q3 krb5kdc[12817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:15.843916 12963 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8a338b3fc724893a8ed3b087497276c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8a338b3fc724893a8ed3b087497276c" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 36523 } }
I20260809 08:52:15.844163 12963 leader_election.cc:302] T 00000000000000000000000000000000 P a8a338b3fc724893a8ed3b087497276c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8a338b3fc724893a8ed3b087497276c; no voters:
I20260809 08:52:15.844470 12963 leader_election.cc:288] T 00000000000000000000000000000000 P a8a338b3fc724893a8ed3b087497276c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:52:15.844551 12968 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8a338b3fc724893a8ed3b087497276c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:52:15.844916 12968 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8a338b3fc724893a8ed3b087497276c [term 1 LEADER]: Becoming Leader. State: Replica: a8a338b3fc724893a8ed3b087497276c, State: Running, Role: LEADER
I20260809 08:52:15.845062 12963 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8a338b3fc724893a8ed3b087497276c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:52:15.845292 12968 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8a338b3fc724893a8ed3b087497276c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8a338b3fc724893a8ed3b087497276c" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 36523 } }
I20260809 08:52:15.846017 12968 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8a338b3fc724893a8ed3b087497276c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8a338b3fc724893a8ed3b087497276c. Latest consensus state: current_term: 1 leader_uuid: "a8a338b3fc724893a8ed3b087497276c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8a338b3fc724893a8ed3b087497276c" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 36523 } } }
I20260809 08:52:15.846252 12968 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8a338b3fc724893a8ed3b087497276c [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:15.846515 12969 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8a338b3fc724893a8ed3b087497276c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a8a338b3fc724893a8ed3b087497276c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8a338b3fc724893a8ed3b087497276c" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 36523 } } }
I20260809 08:52:15.846885 12981 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:52:15.847997 12969 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8a338b3fc724893a8ed3b087497276c [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:15.849148 12981 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:52:15.852963 12981 catalog_manager.cc:1347] Generated new cluster ID: 80d276de927e4de8aaab00dd868e19f2
I20260809 08:52:15.853101 12981 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:52:15.853227 12966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:15.830363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34262 (local address 127.8.154.62:36523)
0809 08:52:15.830850 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:15.830871 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:15.830952 (+ 81us) server_negotiation.cc:407] Connection header received
0809 08:52:15.832118 (+ 1166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:15.832132 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:15.832253 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:15.832512 (+ 259us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:15.834324 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:15.836307 (+ 1983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:15.837527 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:15.838853 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:15.839071 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:15.844149 (+ 5078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:15.844156 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:15.844156 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:15.844171 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:15.848917 (+ 4746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:15.849716 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:15.849730 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:15.849730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:15.850095 (+ 365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:15.850678 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:15.850680 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:15.850680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:15.851541 (+ 861us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:15.852143 (+ 602us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:15.852310 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 08:52:15.852688 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":272,"threads_started":1}
I20260809 08:52:15.871114 12981 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:52:15.872227 12981 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:52:15.885413 12981 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a8a338b3fc724893a8ed3b087497276c: Generated new TSK 0
I20260809 08:52:15.885897 12981 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab.
Entry for principal kudu/127.8.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab.
I20260809 08:52:15.935633 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.1:0
--local_ip_for_outbound_sockets=127.8.154.1
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:36523
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:40909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:16.129256 13002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:16.130054 13002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:16.130169 13002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:16.133847 13002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:16.133960 13002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:16.134049 13002 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.1
I20260809 08:52:16.138643 13002 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:40909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:36523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.1
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:16.139109 13002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:16.139488 13002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:16.151014 13011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:16.151095 13013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:16.151000 13019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:16.151680 13002 server_base.cc:1034] running on GCE node
I20260809 08:52:16.152050 13002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:16.152530 13002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:16.153752 13002 hybrid_clock.cc:648] HybridClock initialized: now 1786265536153731 us; error 50 us; skew 500 ppm
Aug 09 08:52:16 dist-test-slave-t3q3 krb5kdc[12817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265536, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:16.159144 13002 init.cc:377] Logged in from keytab as kudu/127.8.154.1@KRBTEST.COM (short username kudu)
I20260809 08:52:16.160269 13002 webserver.cc:460] Webserver started at http://127.8.154.1:38537/ using document root <none> and password file <none>
I20260809 08:52:16.160720 13002 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:16.160861 13002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:16.161140 13002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:16.163283 13002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-0/data/instance:
uuid: "3124478fd3f149868eb598283bb79742"
format_stamp: "Formatted at 2026-08-09 08:52:16 on dist-test-slave-t3q3"
server_key: "fe8d103b4ffe4c22b4852b16889bb616"
server_key_iv: "7b0cd4b85ec66051747bb475adc5512f"
server_key_version: "encryptionkey@0"
I20260809 08:52:16.164295 13002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance:
uuid: "3124478fd3f149868eb598283bb79742"
format_stamp: "Formatted at 2026-08-09 08:52:16 on dist-test-slave-t3q3"
server_key: "fe8d103b4ffe4c22b4852b16889bb616"
server_key_iv: "7b0cd4b85ec66051747bb475adc5512f"
server_key_version: "encryptionkey@0"
I20260809 08:52:16.168285 13002 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 08:52:16.169842 13030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:16.170197 13002 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:52:16.170358 13002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-0/wal
uuid: "3124478fd3f149868eb598283bb79742"
format_stamp: "Formatted at 2026-08-09 08:52:16 on dist-test-slave-t3q3"
server_key: "fe8d103b4ffe4c22b4852b16889bb616"
server_key_iv: "7b0cd4b85ec66051747bb475adc5512f"
server_key_version: "encryptionkey@0"
I20260809 08:52:16.170483 13002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:16.201483 13002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:16.206243 13002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:16.206553 13002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:16.207106 13002 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:16.207880 13002 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:16.207973 13002 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:16.208102 13002 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:16.208185 13002 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:16.220034 13002 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.1:34843
I20260809 08:52:16.220252 13220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.1:34843 every 8 connection(s)
I20260809 08:52:16.220978 13002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
I20260809 08:52:16.221807 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 13002
I20260809 08:52:16.222077 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance
I20260809 08:52:16.222489 8808 external_mini_cluster.cc:1428] Setting key d4a73a1165d466089eaf013ca2b19c3c
Aug 09 08:52:16 dist-test-slave-t3q3 krb5kdc[12817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265536, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.154.2@KRBTEST.COM; defaulting to no policy
I20260809 08:52:16.241458 12966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:16.222572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:38803 (local address 127.8.154.62:36523)
0809 08:52:16.223019 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:16.223021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:16.223331 (+ 310us) server_negotiation.cc:407] Connection header received
0809 08:52:16.224595 (+ 1264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:16.224596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:16.224616 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:16.224688 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:16.227964 (+ 3276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:16.228879 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:16.230773 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:16.231165 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:16.231317 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:16.235734 (+ 4417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:16.235745 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:16.235745 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:16.235755 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:16.238659 (+ 2904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:16.239497 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:16.239500 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:16.239501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:16.239583 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:16.240034 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:16.240036 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:16.240036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:16.240325 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:16.240491 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:16.240993 (+ 502us) server_negotiation.cc:299] Negotiation successful
0809 08:52:16.241226 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393}
I20260809 08:52:16.241708 13225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:16.222527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:36523 (local address 127.8.154.1:38803)
0809 08:52:16.223004 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:16.223162 (+ 158us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:16.224365 (+ 1203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:16.224878 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:16.224885 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:16.226672 (+ 1787us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:16.227653 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:16.227673 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:16.229071 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:16.229074 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:16.230559 (+ 1485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:16.230563 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:16.231329 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:16.231331 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:16.231665 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:16.232763 (+ 1098us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:16.232779 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:16.235522 (+ 2743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:16.238845 (+ 3323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:16.238864 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:16.238867 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:16.239313 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:16.239707 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:16.239708 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:16.239709 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:16.239891 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:16.240474 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:16.240505 (+ 31us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:16.240713 (+ 208us) client_negotiation.cc:769] Sending connection context
0809 08:52:16.240935 (+ 222us) client_negotiation.cc:240] Negotiation successful
0809 08:52:16.241244 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":246,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.2@KRBTEST.COM" created.
I20260809 08:52:16.242615 13222 heartbeater.cc:347] Connected to a master server at 127.8.154.62:36523
I20260809 08:52:16.242764 13222 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:16.243136 13222 heartbeater.cc:511] Master 127.8.154.62:36523 requested a full tablet report, sending...
I20260809 08:52:16.244151 12898 ts_manager.cc:194] Registered new tserver with Master: 3124478fd3f149868eb598283bb79742 (127.8.154.1:34843)
I20260809 08:52:16.245344 12898 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.1@KRBTEST.COM'} at 127.8.154.1:38803
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab.
Entry for principal kudu/127.8.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab.
I20260809 08:52:16.267663 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.2:0
--local_ip_for_outbound_sockets=127.8.154.2
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:36523
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:40909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:16.395918 13228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:16.396184 13228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:16.396227 13228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:16.397420 13228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:16.397492 13228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:16.397540 13228 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.2
I20260809 08:52:16.399006 13228 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:40909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:36523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.2
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:16.399286 13228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:16.399523 13228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:16.404749 13238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:16.404762 13233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:16.404748 13234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:16.405112 13228 server_base.cc:1034] running on GCE node
I20260809 08:52:16.405344 13228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:16.405593 13228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:16.406741 13228 hybrid_clock.cc:648] HybridClock initialized: now 1786265536406727 us; error 29 us; skew 500 ppm
Aug 09 08:52:16 dist-test-slave-t3q3 krb5kdc[12817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265536, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:16.409610 13228 init.cc:377] Logged in from keytab as kudu/127.8.154.2@KRBTEST.COM (short username kudu)
I20260809 08:52:16.410027 13228 webserver.cc:460] Webserver started at http://127.8.154.2:42469/ using document root <none> and password file <none>
I20260809 08:52:16.410252 13228 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:16.410297 13228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:16.410454 13228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:16.411499 13228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-1/data/instance:
uuid: "2a0bf6a90e3142ff9c54d81ef9645f08"
format_stamp: "Formatted at 2026-08-09 08:52:16 on dist-test-slave-t3q3"
server_key: "34220b736b5e72005da565a19ffaf83f"
server_key_iv: "3742782c9ee08bc25a14430be77d28d0"
server_key_version: "encryptionkey@0"
I20260809 08:52:16.411870 13228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance:
uuid: "2a0bf6a90e3142ff9c54d81ef9645f08"
format_stamp: "Formatted at 2026-08-09 08:52:16 on dist-test-slave-t3q3"
server_key: "34220b736b5e72005da565a19ffaf83f"
server_key_iv: "3742782c9ee08bc25a14430be77d28d0"
server_key_version: "encryptionkey@0"
I20260809 08:52:16.413313 13228 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:16.414094 13252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:16.414299 13228 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:16.414382 13228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-1/wal
uuid: "2a0bf6a90e3142ff9c54d81ef9645f08"
format_stamp: "Formatted at 2026-08-09 08:52:16 on dist-test-slave-t3q3"
server_key: "34220b736b5e72005da565a19ffaf83f"
server_key_iv: "3742782c9ee08bc25a14430be77d28d0"
server_key_version: "encryptionkey@0"
I20260809 08:52:16.414471 13228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:16.453719 13228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:16.455713 13228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:16.455878 13228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:16.456180 13228 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:16.456521 13228 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:16.456578 13228 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:16.456616 13228 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:16.456660 13228 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:16.463089 13228 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.2:45931
I20260809 08:52:16.463232 13433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.2:45931 every 8 connection(s)
I20260809 08:52:16.463510 13228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
I20260809 08:52:16.466889 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 13228
I20260809 08:52:16.467010 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance
I20260809 08:52:16.467216 8808 external_mini_cluster.cc:1428] Setting key 1e0821594174582a778f4f8bb5d0d215
Aug 09 08:52:16 dist-test-slave-t3q3 krb5kdc[12817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265536, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:16.473754 12966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:16.464362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:42617 (local address 127.8.154.62:36523)
0809 08:52:16.464415 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:16.464417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:16.464668 (+ 251us) server_negotiation.cc:407] Connection header received
0809 08:52:16.465216 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:16.465218 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:16.465231 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:16.465298 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:16.466314 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:16.466832 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:16.468168 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:16.468386 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:16.468475 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:16.470683 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:16.470684 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:16.470685 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:16.470692 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:16.472289 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:16.472724 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:16.472726 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:16.472726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:16.472786 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:16.473078 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:16.473079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:16.473080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:16.473201 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:16.473277 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:16.473576 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 08:52:16.473662 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 08:52:16.473960 13437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:16.464345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:36523 (local address 127.8.154.2:42617)
0809 08:52:16.464540 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:16.464578 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:16.465102 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:16.465371 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:16.465375 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:16.465981 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:16.466194 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:16.466199 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:16.466938 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:16.466939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:16.468077 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:16.468080 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:16.468488 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:16.468489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:16.468622 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:16.469175 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:16.469184 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:16.470579 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:16.472376 (+ 1797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:16.472384 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:16.472386 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:16.472622 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:16.472861 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:16.472862 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:16.472862 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:16.472980 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:16.473281 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:16.473289 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:16.473422 (+ 133us) client_negotiation.cc:769] Sending connection context
0809 08:52:16.473550 (+ 128us) client_negotiation.cc:240] Negotiation successful
0809 08:52:16.473714 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":97,"threads_started":1}
I20260809 08:52:16.474471 13435 heartbeater.cc:347] Connected to a master server at 127.8.154.62:36523
I20260809 08:52:16.474567 13435 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:16.474773 13435 heartbeater.cc:511] Master 127.8.154.62:36523 requested a full tablet report, sending...
I20260809 08:52:16.475183 12898 ts_manager.cc:194] Registered new tserver with Master: 2a0bf6a90e3142ff9c54d81ef9645f08 (127.8.154.2:45931)
I20260809 08:52:16.475742 12898 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.2@KRBTEST.COM'} at 127.8.154.2:42617
WARNING: no policy specified for kudu/127.8.154.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab.
Entry for principal kudu/127.8.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab.
I20260809 08:52:16.495627 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.3:0
--local_ip_for_outbound_sockets=127.8.154.3
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:36523
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:40909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:16.579090 13442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:16.579322 13442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:16.579361 13442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:16.580606 13442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:16.580647 13442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:16.580673 13442 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.3
I20260809 08:52:16.582064 13442 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:40909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:36523
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13442
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.3
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:16.582247 13442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:16.582414 13442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:16.587760 13451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:16.587796 13454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:16.587772 13450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:16.588208 13442 server_base.cc:1034] running on GCE node
I20260809 08:52:16.588440 13442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:16.588719 13442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:16.589890 13442 hybrid_clock.cc:648] HybridClock initialized: now 1786265536589879 us; error 41 us; skew 500 ppm
Aug 09 08:52:16 dist-test-slave-t3q3 krb5kdc[12817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265536, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:16.592738 13442 init.cc:377] Logged in from keytab as kudu/127.8.154.3@KRBTEST.COM (short username kudu)
I20260809 08:52:16.593236 13442 webserver.cc:460] Webserver started at http://127.8.154.3:44423/ using document root <none> and password file <none>
I20260809 08:52:16.593529 13442 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:16.593611 13442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:16.593752 13442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:16.594801 13442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-2/data/instance:
uuid: "9bd343314ccb498bbad70a6af962955b"
format_stamp: "Formatted at 2026-08-09 08:52:16 on dist-test-slave-t3q3"
server_key: "df912bb4cf393a62d519605b0dce44c0"
server_key_iv: "1b8c9c0f5f4d78d8909dcbd9a6cac994"
server_key_version: "encryptionkey@0"
I20260809 08:52:16.595181 13442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance:
uuid: "9bd343314ccb498bbad70a6af962955b"
format_stamp: "Formatted at 2026-08-09 08:52:16 on dist-test-slave-t3q3"
server_key: "df912bb4cf393a62d519605b0dce44c0"
server_key_iv: "1b8c9c0f5f4d78d8909dcbd9a6cac994"
server_key_version: "encryptionkey@0"
I20260809 08:52:16.596771 13442 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:52:16.597702 13467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:16.597896 13442 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:52:16.598011 13442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-2/wal
uuid: "9bd343314ccb498bbad70a6af962955b"
format_stamp: "Formatted at 2026-08-09 08:52:16 on dist-test-slave-t3q3"
server_key: "df912bb4cf393a62d519605b0dce44c0"
server_key_iv: "1b8c9c0f5f4d78d8909dcbd9a6cac994"
server_key_version: "encryptionkey@0"
I20260809 08:52:16.598111 13442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:16.628048 13442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:16.630636 13442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:16.630815 13442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:16.631096 13442 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:16.631471 13442 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:16.631515 13442 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:16.631539 13442 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:16.631588 13442 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:16.637753 13442 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.3:40951
I20260809 08:52:16.637814 13647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.3:40951 every 8 connection(s)
I20260809 08:52:16.638144 13442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
I20260809 08:52:16.641444 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 13442
I20260809 08:52:16.641575 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance
I20260809 08:52:16.641775 8808 external_mini_cluster.cc:1428] Setting key f5bb019ee5131048ff334a7127e46eea
Aug 09 08:52:16 dist-test-slave-t3q3 krb5kdc[12817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265536, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:16.648700 12966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:16.638901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:38195 (local address 127.8.154.62:36523)
0809 08:52:16.638976 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:16.638978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:16.639214 (+ 236us) server_negotiation.cc:407] Connection header received
0809 08:52:16.639771 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:16.639773 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:16.639786 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:16.639836 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:16.640894 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:16.641423 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:16.642833 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:16.643101 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:16.643157 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:16.645757 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:16.645759 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:16.645759 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:16.645766 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:16.647322 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:16.647712 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:16.647713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:16.647713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:16.647764 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:16.648039 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:16.648040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:16.648040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:16.648213 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:16.648269 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:16.648568 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 08:52:16.648634 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 08:52:16.648928 13651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:16.638875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:36523 (local address 127.8.154.3:38195)
0809 08:52:16.639075 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:16.639114 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:16.639652 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:16.639934 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:16.639938 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:16.640569 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:16.640780 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:16.640785 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:16.641530 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:16.641532 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:16.642732 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:16.642735 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:16.643182 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:16.643183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:16.643303 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:16.643870 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:16.643879 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:16.645650 (+ 1771us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:16.647398 (+ 1748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:16.647405 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:16.647406 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:16.647642 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:16.647844 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:16.647844 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:16.647845 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:16.647952 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:16.648287 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:16.648294 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:16.648433 (+ 139us) client_negotiation.cc:769] Sending connection context
0809 08:52:16.648536 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 08:52:16.648682 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":95,"threads_started":1}
I20260809 08:52:16.649605 13648 heartbeater.cc:347] Connected to a master server at 127.8.154.62:36523
I20260809 08:52:16.649701 13648 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:16.649915 13648 heartbeater.cc:511] Master 127.8.154.62:36523 requested a full tablet report, sending...
I20260809 08:52:16.650368 12898 ts_manager.cc:194] Registered new tserver with Master: 9bd343314ccb498bbad70a6af962955b (127.8.154.3:40951)
I20260809 08:52:16.650779 12898 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.3@KRBTEST.COM'} at 127.8.154.3:38195
I20260809 08:52:16.653865 8808 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 08:52:16.667994 12966 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:16.666820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34270 (local address 127.8.154.62:36523)
0809 08:52:16.666951 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:16.666954 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:16.667065 (+ 111us) server_negotiation.cc:407] Connection header received
0809 08:52:16.667187 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:16.667188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:16.667201 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:16.667279 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:16.667909 (+ 630us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34270: BlockingRecv error: recv got EOF from 127.0.0.1:34270 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
I20260809 08:52:16.668901 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 13002
I20260809 08:52:16.680399 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 13228
I20260809 08:52:16.704209 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 13442
I20260809 08:52:16.712670 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 12847
2026-08-09T08:52:16Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2895 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:52:16 dist-test-slave-t3q3 krb5kdc[13666](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:52:16 dist-test-slave-t3q3 krb5kdc[13666](info): set up 2 sockets
Aug 09 08:52:16 dist-test-slave-t3q3 krb5kdc[13666](info): commencing operation
krb5kdc: starting...
W20260809 08:52:17.981503 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.240s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:52:18 dist-test-slave-t3q3 krb5kdc[13666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:52:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:52:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
I20260809 08:52:18.103451 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:38839
--webserver_interface=127.8.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:42805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:38839
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:18.185194 13690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:18.185400 13690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:18.185422 13690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:18.186602 13690 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:52:18.186645 13690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:18.186659 13690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:18.186672 13690 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:52:18.186684 13690 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:52:18.188297 13690 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:42805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:38839
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:38839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:18.188555 13690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:18.188781 13690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:18.193723 13707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:18.193794 13710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:18.193805 13700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:18.193941 13690 server_base.cc:1034] running on GCE node
I20260809 08:52:18.194199 13690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:18.194530 13690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:18.195709 13690 hybrid_clock.cc:648] HybridClock initialized: now 1786265538195698 us; error 32 us; skew 500 ppm
Aug 09 08:52:18 dist-test-slave-t3q3 krb5kdc[13666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265538, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:18.201685 13690 init.cc:377] Logged in from keytab as kudu/127.8.154.62@KRBTEST.COM (short username kudu)
I20260809 08:52:18.202267 13690 webserver.cc:460] Webserver started at http://127.8.154.62:33165/ using document root <none> and password file <none>
I20260809 08:52:18.202519 13690 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:18.202571 13690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:18.202734 13690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:18.203850 13690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "3dfa7d30b69c41f492c70bc3df90b90f"
format_stamp: "Formatted at 2026-08-09 08:52:18 on dist-test-slave-t3q3"
server_key: "61e41fc8e92b1d95bcdfd64d8b5a351a"
server_key_iv: "8e0f678dbd5834a745df87386efe28e9"
server_key_version: "encryptionkey@0"
I20260809 08:52:18.204263 13690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "3dfa7d30b69c41f492c70bc3df90b90f"
format_stamp: "Formatted at 2026-08-09 08:52:18 on dist-test-slave-t3q3"
server_key: "61e41fc8e92b1d95bcdfd64d8b5a351a"
server_key_iv: "8e0f678dbd5834a745df87386efe28e9"
server_key_version: "encryptionkey@0"
I20260809 08:52:18.205679 13690 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:52:18.206444 13720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:18.206673 13690 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:52:18.206770 13690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "3dfa7d30b69c41f492c70bc3df90b90f"
format_stamp: "Formatted at 2026-08-09 08:52:18 on dist-test-slave-t3q3"
server_key: "61e41fc8e92b1d95bcdfd64d8b5a351a"
server_key_iv: "8e0f678dbd5834a745df87386efe28e9"
server_key_version: "encryptionkey@0"
I20260809 08:52:18.206826 13690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:18.236475 13690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:18.238889 13690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:18.239009 13690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:18.243068 13690 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:38839
I20260809 08:52:18.243165 13808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:38839 every 8 connection(s)
I20260809 08:52:18.243496 13690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:52:18.244457 13809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:18.246775 13809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3dfa7d30b69c41f492c70bc3df90b90f: Bootstrap starting.
I20260809 08:52:18.247396 13809 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3dfa7d30b69c41f492c70bc3df90b90f: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:18.247709 13809 log.cc:824] T 00000000000000000000000000000000 P 3dfa7d30b69c41f492c70bc3df90b90f: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:18.248482 13809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3dfa7d30b69c41f492c70bc3df90b90f: No bootstrap required, opened a new log
I20260809 08:52:18.248473 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 13690
I20260809 08:52:18.248570 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:52:18.248764 8808 external_mini_cluster.cc:1428] Setting key 4bce35e2c30137bf96f5fc67a1701f30
I20260809 08:52:18.251454 13809 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3dfa7d30b69c41f492c70bc3df90b90f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dfa7d30b69c41f492c70bc3df90b90f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 38839 } }
I20260809 08:52:18.251629 13809 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3dfa7d30b69c41f492c70bc3df90b90f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:18.251663 13809 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3dfa7d30b69c41f492c70bc3df90b90f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3dfa7d30b69c41f492c70bc3df90b90f, State: Initialized, Role: FOLLOWER
I20260809 08:52:18.251801 13809 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3dfa7d30b69c41f492c70bc3df90b90f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dfa7d30b69c41f492c70bc3df90b90f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 38839 } }
I20260809 08:52:18.251876 13809 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3dfa7d30b69c41f492c70bc3df90b90f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:52:18.251904 13809 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3dfa7d30b69c41f492c70bc3df90b90f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:52:18.251940 13809 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3dfa7d30b69c41f492c70bc3df90b90f [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:18.252617 13809 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3dfa7d30b69c41f492c70bc3df90b90f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dfa7d30b69c41f492c70bc3df90b90f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 38839 } }
I20260809 08:52:18.252710 13809 leader_election.cc:302] T 00000000000000000000000000000000 P 3dfa7d30b69c41f492c70bc3df90b90f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3dfa7d30b69c41f492c70bc3df90b90f; no voters:
I20260809 08:52:18.252905 13809 leader_election.cc:288] T 00000000000000000000000000000000 P 3dfa7d30b69c41f492c70bc3df90b90f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:52:18.252967 13821 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3dfa7d30b69c41f492c70bc3df90b90f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:52:18.253135 13821 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3dfa7d30b69c41f492c70bc3df90b90f [term 1 LEADER]: Becoming Leader. State: Replica: 3dfa7d30b69c41f492c70bc3df90b90f, State: Running, Role: LEADER
I20260809 08:52:18.253204 13809 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3dfa7d30b69c41f492c70bc3df90b90f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:52:18.253242 13821 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3dfa7d30b69c41f492c70bc3df90b90f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dfa7d30b69c41f492c70bc3df90b90f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 38839 } }
I20260809 08:52:18.253602 13822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3dfa7d30b69c41f492c70bc3df90b90f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3dfa7d30b69c41f492c70bc3df90b90f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dfa7d30b69c41f492c70bc3df90b90f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 38839 } } }
I20260809 08:52:18.253697 13822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3dfa7d30b69c41f492c70bc3df90b90f [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:18.253961 13823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3dfa7d30b69c41f492c70bc3df90b90f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3dfa7d30b69c41f492c70bc3df90b90f. Latest consensus state: current_term: 1 leader_uuid: "3dfa7d30b69c41f492c70bc3df90b90f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dfa7d30b69c41f492c70bc3df90b90f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 38839 } } }
I20260809 08:52:18.254098 13823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3dfa7d30b69c41f492c70bc3df90b90f [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:18.254261 13829 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:52:18.254812 13829 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 08:52:18 dist-test-slave-t3q3 krb5kdc[13666](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:18.257220 13829 catalog_manager.cc:1347] Generated new cluster ID: 8b5fe6098b574eb9a9f74eede87a5331
I20260809 08:52:18.257284 13829 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:52:18.260998 13820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:18.249315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57986 (local address 127.8.154.62:38839)
0809 08:52:18.249522 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:18.249532 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:18.249573 (+ 41us) server_negotiation.cc:407] Connection header received
0809 08:52:18.250034 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:18.250043 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:18.250105 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:18.250257 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:18.250888 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:18.252283 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:18.253313 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:18.253954 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:18.254060 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:18.257185 (+ 3125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:18.257190 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:18.257190 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:18.257202 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:18.259138 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:18.259499 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:18.259502 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:18.259502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:18.259603 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:18.259773 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:18.259774 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:18.259774 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:18.260037 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:18.260207 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:18.260670 (+ 463us) server_negotiation.cc:299] Negotiation successful
0809 08:52:18.260798 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":82,"threads_started":1}
I20260809 08:52:18.261948 8808 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:52:18.269318 13829 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:52:18.269906 13829 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:52:18.275182 13829 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3dfa7d30b69c41f492c70bc3df90b90f: Generated new TSK 0
I20260809 08:52:18.275365 13829 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:52:18.282334 13820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:18.278463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58000 (local address 127.8.154.62:38839)
0809 08:52:18.278528 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:18.278530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:18.278618 (+ 88us) server_negotiation.cc:407] Connection header received
0809 08:52:18.278652 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:18.278653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:18.278665 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:18.278747 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:18.279083 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:18.279861 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:18.280670 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:18.281000 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:18.281084 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:18.281798 (+ 714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:18.281800 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:18.281800 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:52:18.282065 (+ 265us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:18.282085 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:52:18.282093 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:18.282158 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:18.282230 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 08:52:18.282257 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 08:52:18.284901 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 13690
2026-08-09T08:52:18Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1566 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:52:18 dist-test-slave-t3q3 krb5kdc[13869](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:52:18 dist-test-slave-t3q3 krb5kdc[13869](info): set up 2 sockets
Aug 09 08:52:18 dist-test-slave-t3q3 krb5kdc[13869](info): commencing operation
krb5kdc: starting...
W20260809 08:52:19.552734 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:52:19 dist-test-slave-t3q3 krb5kdc[13869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:52:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:52:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
I20260809 08:52:19.678282 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:35495
--webserver_interface=127.8.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:40833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:35495
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:19.760253 13896 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:19.760497 13896 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:19.760540 13896 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:19.761677 13896 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:52:19.761745 13896 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:19.761770 13896 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:19.761791 13896 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:52:19.761817 13896 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:52:19.763294 13896 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:40833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:35495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:35495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13896
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:19.763568 13896 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:19.763793 13896 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:19.768671 13905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:19.768671 13915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:19.768739 13906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:19.769027 13896 server_base.cc:1034] running on GCE node
I20260809 08:52:19.769304 13896 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:19.769601 13896 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:19.770787 13896 hybrid_clock.cc:648] HybridClock initialized: now 1786265539770766 us; error 30 us; skew 500 ppm
Aug 09 08:52:19 dist-test-slave-t3q3 krb5kdc[13869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265539, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:19.773546 13896 init.cc:377] Logged in from keytab as kudu/127.8.154.62@KRBTEST.COM (short username kudu)
I20260809 08:52:19.773923 13896 webserver.cc:460] Webserver started at http://127.8.154.62:34873/ using document root <none> and password file <none>
I20260809 08:52:19.774129 13896 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:19.774170 13896 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:19.774258 13896 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:19.775238 13896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "42c5a322a0e94f778559cf9ac917ad43"
format_stamp: "Formatted at 2026-08-09 08:52:19 on dist-test-slave-t3q3"
server_key: "4dfe4e331c97f55b0fc0d4bf4dd5d42f"
server_key_iv: "23ce9f2aff910ca81beeca89b170d135"
server_key_version: "encryptionkey@0"
I20260809 08:52:19.775620 13896 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "42c5a322a0e94f778559cf9ac917ad43"
format_stamp: "Formatted at 2026-08-09 08:52:19 on dist-test-slave-t3q3"
server_key: "4dfe4e331c97f55b0fc0d4bf4dd5d42f"
server_key_iv: "23ce9f2aff910ca81beeca89b170d135"
server_key_version: "encryptionkey@0"
I20260809 08:52:19.777010 13896 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:52:19.777755 13925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:19.777952 13896 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:19.778061 13896 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "42c5a322a0e94f778559cf9ac917ad43"
format_stamp: "Formatted at 2026-08-09 08:52:19 on dist-test-slave-t3q3"
server_key: "4dfe4e331c97f55b0fc0d4bf4dd5d42f"
server_key_iv: "23ce9f2aff910ca81beeca89b170d135"
server_key_version: "encryptionkey@0"
I20260809 08:52:19.778151 13896 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:19.809170 13896 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:19.811321 13896 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:19.811486 13896 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:19.815692 13896 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:35495
I20260809 08:52:19.815745 14016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:35495 every 8 connection(s)
I20260809 08:52:19.816180 13896 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:52:19.817404 14019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:19.819782 14019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42c5a322a0e94f778559cf9ac917ad43: Bootstrap starting.
I20260809 08:52:19.820421 14019 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42c5a322a0e94f778559cf9ac917ad43: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:19.820710 14019 log.cc:824] T 00000000000000000000000000000000 P 42c5a322a0e94f778559cf9ac917ad43: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:19.821449 14019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42c5a322a0e94f778559cf9ac917ad43: No bootstrap required, opened a new log
I20260809 08:52:19.823246 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 13896
I20260809 08:52:19.823119 14019 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42c5a322a0e94f778559cf9ac917ad43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c5a322a0e94f778559cf9ac917ad43" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 35495 } }
I20260809 08:52:19.823308 14019 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42c5a322a0e94f778559cf9ac917ad43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:19.823357 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:52:19.823350 14019 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42c5a322a0e94f778559cf9ac917ad43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42c5a322a0e94f778559cf9ac917ad43, State: Initialized, Role: FOLLOWER
I20260809 08:52:19.823546 8808 external_mini_cluster.cc:1428] Setting key 67d4641936bddf7125eafe9567fffe05
I20260809 08:52:19.823518 14019 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42c5a322a0e94f778559cf9ac917ad43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c5a322a0e94f778559cf9ac917ad43" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 35495 } }
I20260809 08:52:19.823601 14019 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42c5a322a0e94f778559cf9ac917ad43 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:52:19.823635 14019 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42c5a322a0e94f778559cf9ac917ad43 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:52:19.823679 14019 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42c5a322a0e94f778559cf9ac917ad43 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:19.824388 14019 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42c5a322a0e94f778559cf9ac917ad43 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c5a322a0e94f778559cf9ac917ad43" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 35495 } }
I20260809 08:52:19.824482 14019 leader_election.cc:302] T 00000000000000000000000000000000 P 42c5a322a0e94f778559cf9ac917ad43 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42c5a322a0e94f778559cf9ac917ad43; no voters:
I20260809 08:52:19.824687 14019 leader_election.cc:288] T 00000000000000000000000000000000 P 42c5a322a0e94f778559cf9ac917ad43 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:52:19.824715 14026 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42c5a322a0e94f778559cf9ac917ad43 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:52:19.824893 14026 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42c5a322a0e94f778559cf9ac917ad43 [term 1 LEADER]: Becoming Leader. State: Replica: 42c5a322a0e94f778559cf9ac917ad43, State: Running, Role: LEADER
I20260809 08:52:19.825048 14019 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42c5a322a0e94f778559cf9ac917ad43 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:52:19.825037 14026 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42c5a322a0e94f778559cf9ac917ad43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c5a322a0e94f778559cf9ac917ad43" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 35495 } }
I20260809 08:52:19.826659 14029 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42c5a322a0e94f778559cf9ac917ad43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42c5a322a0e94f778559cf9ac917ad43. Latest consensus state: current_term: 1 leader_uuid: "42c5a322a0e94f778559cf9ac917ad43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c5a322a0e94f778559cf9ac917ad43" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 35495 } } }
I20260809 08:52:19.826812 14031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42c5a322a0e94f778559cf9ac917ad43 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42c5a322a0e94f778559cf9ac917ad43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42c5a322a0e94f778559cf9ac917ad43" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 35495 } } }
I20260809 08:52:19.827289 14029 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42c5a322a0e94f778559cf9ac917ad43 [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:19.827308 14031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42c5a322a0e94f778559cf9ac917ad43 [sys.catalog]: This master's current role is: LEADER
W20260809 08:52:19.827173 14051 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 42c5a322a0e94f778559cf9ac917ad43: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:52:19.827447 14051 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:52:19.827615 14054 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:52:19.827814 14054 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:52:19.829536 14054 catalog_manager.cc:1347] Generated new cluster ID: ad84eb90df6e43ca8b43888a1852e482
I20260809 08:52:19.829603 14054 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 08:52:19 dist-test-slave-t3q3 krb5kdc[13869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:19.836647 14028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:19.824087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35416 (local address 127.8.154.62:35495)
0809 08:52:19.824768 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:19.824778 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:19.824812 (+ 34us) server_negotiation.cc:407] Connection header received
0809 08:52:19.824939 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:19.824948 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:19.824998 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:19.825169 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:19.826084 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:19.827604 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:19.828696 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:19.829312 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:19.829435 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:19.832225 (+ 2790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:19.832230 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:19.832230 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:19.832243 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:19.834887 (+ 2644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:19.835321 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:19.835324 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:19.835325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:19.835452 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:19.835668 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:19.835669 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:19.835669 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:19.835985 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:19.836189 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:19.836274 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 08:52:19.836421 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":589,"threads_started":1}
I20260809 08:52:19.842044 14054 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:52:19.842698 14054 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:52:19.844570 14054 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42c5a322a0e94f778559cf9ac917ad43: Generated new TSK 0
I20260809 08:52:19.844738 14054 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab.
Entry for principal kudu/127.8.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab.
I20260809 08:52:19.874773 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.1:0
--local_ip_for_outbound_sockets=127.8.154.1
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:35495
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:40833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:19.961447 14062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:19.961650 14062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:19.961673 14062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:19.962821 14062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:19.962865 14062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:19.962893 14062 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.1
I20260809 08:52:19.964398 14062 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:40833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:35495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.1
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:19.964650 14062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:19.964896 14062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:19.970211 14081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:19.970201 14076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:19.970228 14083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:19.970726 14062 server_base.cc:1034] running on GCE node
I20260809 08:52:19.970940 14062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:19.971161 14062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:19.972313 14062 hybrid_clock.cc:648] HybridClock initialized: now 1786265539972295 us; error 30 us; skew 500 ppm
Aug 09 08:52:19 dist-test-slave-t3q3 krb5kdc[13869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265539, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:19.975052 14062 init.cc:377] Logged in from keytab as kudu/127.8.154.1@KRBTEST.COM (short username kudu)
I20260809 08:52:19.975593 14062 webserver.cc:460] Webserver started at http://127.8.154.1:37105/ using document root <none> and password file <none>
I20260809 08:52:19.975844 14062 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:19.975895 14062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:19.976045 14062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:19.977269 14062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-0/data/instance:
uuid: "f7c36c4e9fcd4b14903e3e12532d241c"
format_stamp: "Formatted at 2026-08-09 08:52:19 on dist-test-slave-t3q3"
server_key: "881f209a6579cb519fe66234becff86c"
server_key_iv: "0f8db4349eeee847d9227af002518a91"
server_key_version: "encryptionkey@0"
I20260809 08:52:19.977689 14062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance:
uuid: "f7c36c4e9fcd4b14903e3e12532d241c"
format_stamp: "Formatted at 2026-08-09 08:52:19 on dist-test-slave-t3q3"
server_key: "881f209a6579cb519fe66234becff86c"
server_key_iv: "0f8db4349eeee847d9227af002518a91"
server_key_version: "encryptionkey@0"
I20260809 08:52:19.979175 14062 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:52:19.980129 14097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:19.980383 14062 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:19.980486 14062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-0/wal
uuid: "f7c36c4e9fcd4b14903e3e12532d241c"
format_stamp: "Formatted at 2026-08-09 08:52:19 on dist-test-slave-t3q3"
server_key: "881f209a6579cb519fe66234becff86c"
server_key_iv: "0f8db4349eeee847d9227af002518a91"
server_key_version: "encryptionkey@0"
I20260809 08:52:19.980552 14062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:20.000203 14062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:20.003059 14062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:20.003183 14062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:20.003377 14062 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:20.003737 14062 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:20.003777 14062 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:20.003801 14062 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:20.003814 14062 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:20.009708 14062 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.1:43047
I20260809 08:52:20.009789 14289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.1:43047 every 8 connection(s)
I20260809 08:52:20.010120 14062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
I20260809 08:52:20.010397 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 14062
I20260809 08:52:20.010514 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance
I20260809 08:52:20.010752 8808 external_mini_cluster.cc:1428] Setting key a2350ab04f53e17bb5cc481e94e5d246
Aug 09 08:52:20 dist-test-slave-t3q3 krb5kdc[13869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265539, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.154.2@KRBTEST.COM; defaulting to no policy
I20260809 08:52:20.020844 14028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:20.011050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:52827 (local address 127.8.154.62:35495)
0809 08:52:20.011370 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:20.011371 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:20.011383 (+ 12us) server_negotiation.cc:407] Connection header received
0809 08:52:20.011905 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:20.011906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:20.011922 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:20.011994 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:20.013047 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:20.013610 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:20.014708 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:20.014985 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:20.015077 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:20.017741 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:20.017743 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:20.017743 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:20.017750 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:20.019204 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:20.019741 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:20.019742 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:20.019742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:20.019796 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:20.020141 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:20.020142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:20.020142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:20.020251 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:20.020307 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:20.020699 (+ 392us) server_negotiation.cc:299] Negotiation successful
0809 08:52:20.020777 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 08:52:20.021013 14294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:20.010928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:35495 (local address 127.8.154.1:52827)
0809 08:52:20.011215 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:20.011262 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:20.011802 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:20.012117 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:20.012122 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:20.012712 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:20.012908 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:20.012913 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:20.013703 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:20.013704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:20.014553 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:20.014556 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:20.015090 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:20.015091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:20.015229 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:20.015749 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:20.015758 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:20.017643 (+ 1885us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:20.019297 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:20.019305 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:20.019307 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:20.019659 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:20.019877 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:20.019878 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:20.019879 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:20.020024 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:20.020330 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:20.020338 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:20.020518 (+ 180us) client_negotiation.cc:769] Sending connection context
0809 08:52:20.020650 (+ 132us) client_negotiation.cc:240] Negotiation successful
0809 08:52:20.020803 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":98,"threads_started":1}
I20260809 08:52:20.021545 14290 heartbeater.cc:347] Connected to a master server at 127.8.154.62:35495
I20260809 08:52:20.021639 14290 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:20.021783 14290 heartbeater.cc:511] Master 127.8.154.62:35495 requested a full tablet report, sending...
I20260809 08:52:20.022306 13957 ts_manager.cc:194] Registered new tserver with Master: f7c36c4e9fcd4b14903e3e12532d241c (127.8.154.1:43047)
I20260809 08:52:20.022867 13957 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.1@KRBTEST.COM'} at 127.8.154.1:52827
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab.
Entry for principal kudu/127.8.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab.
I20260809 08:52:20.038452 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.2:0
--local_ip_for_outbound_sockets=127.8.154.2
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:35495
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:40833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:20.121414 14300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:20.121686 14300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:20.121726 14300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:20.122907 14300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:20.122987 14300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:20.123028 14300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.2
I20260809 08:52:20.124522 14300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:40833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:35495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.2
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:20.124827 14300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:20.125067 14300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:20.130465 14311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:20.130482 14313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:20.130568 14308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:20.130829 14300 server_base.cc:1034] running on GCE node
I20260809 08:52:20.131011 14300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:20.131237 14300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:20.132392 14300 hybrid_clock.cc:648] HybridClock initialized: now 1786265540132382 us; error 30 us; skew 500 ppm
Aug 09 08:52:20 dist-test-slave-t3q3 krb5kdc[13869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265540, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:20.135118 14300 init.cc:377] Logged in from keytab as kudu/127.8.154.2@KRBTEST.COM (short username kudu)
I20260809 08:52:20.135598 14300 webserver.cc:460] Webserver started at http://127.8.154.2:38465/ using document root <none> and password file <none>
I20260809 08:52:20.135825 14300 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:20.135871 14300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:20.136005 14300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:20.137132 14300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-1/data/instance:
uuid: "877f0879f8d34d699812fe8811a62231"
format_stamp: "Formatted at 2026-08-09 08:52:20 on dist-test-slave-t3q3"
server_key: "b7607ab709a336557dcd8207612eb10a"
server_key_iv: "bea462a87e5a5081343d59acc2028278"
server_key_version: "encryptionkey@0"
I20260809 08:52:20.137502 14300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance:
uuid: "877f0879f8d34d699812fe8811a62231"
format_stamp: "Formatted at 2026-08-09 08:52:20 on dist-test-slave-t3q3"
server_key: "b7607ab709a336557dcd8207612eb10a"
server_key_iv: "bea462a87e5a5081343d59acc2028278"
server_key_version: "encryptionkey@0"
I20260809 08:52:20.138862 14300 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:52:20.139629 14326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:20.139803 14300 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:52:20.139912 14300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-1/wal
uuid: "877f0879f8d34d699812fe8811a62231"
format_stamp: "Formatted at 2026-08-09 08:52:20 on dist-test-slave-t3q3"
server_key: "b7607ab709a336557dcd8207612eb10a"
server_key_iv: "bea462a87e5a5081343d59acc2028278"
server_key_version: "encryptionkey@0"
I20260809 08:52:20.139994 14300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:20.150625 14300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:20.152999 14300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:20.153170 14300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:20.153409 14300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:20.153725 14300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:20.153779 14300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:20.153811 14300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:20.153841 14300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:20.160033 14300 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.2:41811
I20260809 08:52:20.160126 14516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.2:41811 every 8 connection(s)
I20260809 08:52:20.160435 14300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
I20260809 08:52:20.163746 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 14300
I20260809 08:52:20.163838 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance
I20260809 08:52:20.164081 8808 external_mini_cluster.cc:1428] Setting key 9d4a509d23891c7f57e7a82d4b049b20
Aug 09 08:52:20 dist-test-slave-t3q3 krb5kdc[13869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265540, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:20.170244 14028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:20.161181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:58495 (local address 127.8.154.62:35495)
0809 08:52:20.161248 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:20.161250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:20.161444 (+ 194us) server_negotiation.cc:407] Connection header received
0809 08:52:20.161934 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:20.161935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:20.161947 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:20.162019 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:20.162916 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:20.163446 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:20.164767 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:20.164992 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:20.165091 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:20.167356 (+ 2265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:20.167358 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:20.167358 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:20.167366 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:20.168819 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:20.169247 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:20.169248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:20.169248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:20.169314 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:20.169579 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:20.169580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:20.169580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:20.169701 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:20.169776 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:20.170055 (+ 279us) server_negotiation.cc:299] Negotiation successful
0809 08:52:20.170150 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 08:52:20.170444 14521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:20.161156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:35495 (local address 127.8.154.2:58495)
0809 08:52:20.161315 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:20.161356 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:20.161822 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:20.162076 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:20.162079 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:20.162624 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:20.162809 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:20.162814 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:20.163536 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:20.163536 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:20.164661 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:20.164664 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:20.165100 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:20.165101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:20.165286 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:20.165841 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:20.165850 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:20.167259 (+ 1409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:20.168924 (+ 1665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:20.168932 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:20.168934 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:20.169167 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:20.169388 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:20.169389 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:20.169389 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:20.169506 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:20.169778 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:20.169786 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:20.169923 (+ 137us) client_negotiation.cc:769] Sending connection context
0809 08:52:20.170033 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 08:52:20.170198 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":90,"threads_started":1}
I20260809 08:52:20.170974 14517 heartbeater.cc:347] Connected to a master server at 127.8.154.62:35495
I20260809 08:52:20.171056 14517 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:20.171259 14517 heartbeater.cc:511] Master 127.8.154.62:35495 requested a full tablet report, sending...
I20260809 08:52:20.171708 13957 ts_manager.cc:194] Registered new tserver with Master: 877f0879f8d34d699812fe8811a62231 (127.8.154.2:41811)
I20260809 08:52:20.172263 13957 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.2@KRBTEST.COM'} at 127.8.154.2:58495
WARNING: no policy specified for kudu/127.8.154.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab.
Entry for principal kudu/127.8.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab.
I20260809 08:52:20.191097 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.3:0
--local_ip_for_outbound_sockets=127.8.154.3
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:35495
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:40833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:20.275194 14525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:20.275381 14525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:20.275405 14525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:20.276582 14525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:20.276623 14525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:20.276649 14525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.3
I20260809 08:52:20.277966 14525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:40833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:35495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.3
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:20.278139 14525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:20.278308 14525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:20.283388 14534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:20.283419 14533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:20.283484 14542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:20.283435 14525 server_base.cc:1034] running on GCE node
I20260809 08:52:20.283754 14525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:20.284108 14525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:20.285280 14525 hybrid_clock.cc:648] HybridClock initialized: now 1786265540285264 us; error 27 us; skew 500 ppm
Aug 09 08:52:20 dist-test-slave-t3q3 krb5kdc[13869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265540, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:20.287874 14525 init.cc:377] Logged in from keytab as kudu/127.8.154.3@KRBTEST.COM (short username kudu)
I20260809 08:52:20.288399 14525 webserver.cc:460] Webserver started at http://127.8.154.3:32913/ using document root <none> and password file <none>
I20260809 08:52:20.288638 14525 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:20.288709 14525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:20.288873 14525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:20.289938 14525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-2/data/instance:
uuid: "6074560669e9443cbe7ffd21c2f9da94"
format_stamp: "Formatted at 2026-08-09 08:52:20 on dist-test-slave-t3q3"
server_key: "8764046d4825bacc8172a33d791e780c"
server_key_iv: "f574f943e4e8b72130961519f2b3c43d"
server_key_version: "encryptionkey@0"
I20260809 08:52:20.290335 14525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance:
uuid: "6074560669e9443cbe7ffd21c2f9da94"
format_stamp: "Formatted at 2026-08-09 08:52:20 on dist-test-slave-t3q3"
server_key: "8764046d4825bacc8172a33d791e780c"
server_key_iv: "f574f943e4e8b72130961519f2b3c43d"
server_key_version: "encryptionkey@0"
I20260809 08:52:20.291781 14525 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:52:20.292635 14552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:20.292809 14525 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:20.292914 14525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-2/wal
uuid: "6074560669e9443cbe7ffd21c2f9da94"
format_stamp: "Formatted at 2026-08-09 08:52:20 on dist-test-slave-t3q3"
server_key: "8764046d4825bacc8172a33d791e780c"
server_key_iv: "f574f943e4e8b72130961519f2b3c43d"
server_key_version: "encryptionkey@0"
I20260809 08:52:20.293046 14525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:20.312153 14525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:20.314138 14525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:20.314312 14525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:20.314574 14525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:20.314920 14525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:20.314976 14525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:20.315029 14525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:20.315073 14525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:20.321280 14525 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.3:38435
I20260809 08:52:20.321321 14738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.3:38435 every 8 connection(s)
I20260809 08:52:20.321645 14525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
I20260809 08:52:20.326339 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 14525
I20260809 08:52:20.326467 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance
I20260809 08:52:20.326725 8808 external_mini_cluster.cc:1428] Setting key ad4e2e47620f90e6ab58891753345226
Aug 09 08:52:20 dist-test-slave-t3q3 krb5kdc[13869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265540, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:20.331494 14028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:20.322444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:47009 (local address 127.8.154.62:35495)
0809 08:52:20.322506 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:20.322509 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:20.322716 (+ 207us) server_negotiation.cc:407] Connection header received
0809 08:52:20.323288 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:20.323289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:20.323303 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:20.323350 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:20.324370 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:20.324980 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:20.326193 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:20.326527 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:20.326607 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:20.329153 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:20.329154 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:20.329154 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:20.329161 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:20.330337 (+ 1176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:20.330713 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:20.330714 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:20.330714 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:20.330766 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:20.330981 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:20.330981 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:20.330981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:20.331089 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:20.331139 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:20.331370 (+ 231us) server_negotiation.cc:299] Negotiation successful
0809 08:52:20.331432 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 08:52:20.331660 14742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:20.322412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:35495 (local address 127.8.154.3:47009)
0809 08:52:20.322596 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:20.322633 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:20.323169 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:20.323433 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:20.323437 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:20.324009 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:20.324266 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:20.324272 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:20.325081 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:20.325082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:20.326098 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:20.326100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:20.326620 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:20.326621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:20.326851 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:20.327441 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:20.327451 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:20.329069 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:20.330407 (+ 1338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:20.330413 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:20.330415 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:20.330648 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:20.330825 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:20.330826 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:20.330826 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:20.330926 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:20.331144 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:20.331149 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:20.331260 (+ 111us) client_negotiation.cc:769] Sending connection context
0809 08:52:20.331341 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 08:52:20.331456 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":104,"threads_started":1}
I20260809 08:52:20.332149 14740 heartbeater.cc:347] Connected to a master server at 127.8.154.62:35495
I20260809 08:52:20.332242 14740 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:20.332384 14740 heartbeater.cc:511] Master 127.8.154.62:35495 requested a full tablet report, sending...
I20260809 08:52:20.332741 13957 ts_manager.cc:194] Registered new tserver with Master: 6074560669e9443cbe7ffd21c2f9da94 (127.8.154.3:38435)
I20260809 08:52:20.333141 13957 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.3@KRBTEST.COM'} at 127.8.154.3:47009
I20260809 08:52:20.338915 8808 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 08:52:20 dist-test-slave-t3q3 krb5kdc[13869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265540, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 08:52:20 dist-test-slave-t3q3 krb5kdc[13869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265540, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:20.363117 14028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:20.355939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35418 (local address 127.8.154.62:35495)
0809 08:52:20.356008 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:20.356010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:20.356163 (+ 153us) server_negotiation.cc:407] Connection header received
0809 08:52:20.356228 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:20.356229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:20.356238 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:20.356290 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:20.356858 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:20.357344 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:20.358166 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:20.358353 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:20.358408 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:20.360677 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:20.360679 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:20.360679 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:20.360686 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:20.362083 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:20.362397 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:20.362398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:20.362398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:20.362450 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:20.362613 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:20.362614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:20.362614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:20.362755 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:20.362837 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:20.362936 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 08:52:20.362997 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260809 08:52:20.363406 13957 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:35418
I20260809 08:52:20.364123 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 14062
I20260809 08:52:20.383803 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 14300
I20260809 08:52:20.394542 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 14525
I20260809 08:52:20.409811 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 13896
2026-08-09T08:52:20Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2123 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:52:20 dist-test-slave-t3q3 krb5kdc[14765](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:52:20 dist-test-slave-t3q3 krb5kdc[14765](info): set up 2 sockets
Aug 09 08:52:20 dist-test-slave-t3q3 krb5kdc[14765](info): commencing operation
krb5kdc: starting...
W20260809 08:52:21.673200 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:52:21 dist-test-slave-t3q3 krb5kdc[14765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265541, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:52:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:52:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
I20260809 08:52:21.796584 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:35589
--webserver_interface=127.8.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:41241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:35589
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:21.879813 14789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:21.880012 14789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:21.880034 14789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:21.881310 14789 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:52:21.881351 14789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:21.881366 14789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:21.881381 14789 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:52:21.881392 14789 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:52:21.882899 14789 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:41241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:35589
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:35589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:21.883147 14789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:21.883388 14789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:21.888383 14797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:21.888399 14799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:21.888399 14802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:21.888659 14789 server_base.cc:1034] running on GCE node
I20260809 08:52:21.888949 14789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:21.889235 14789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:21.890403 14789 hybrid_clock.cc:648] HybridClock initialized: now 1786265541890390 us; error 49 us; skew 500 ppm
Aug 09 08:52:21 dist-test-slave-t3q3 krb5kdc[14765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265541, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:21.893107 14789 init.cc:377] Logged in from keytab as kudu/127.8.154.62@KRBTEST.COM (short username kudu)
I20260809 08:52:21.893625 14789 webserver.cc:460] Webserver started at http://127.8.154.62:45967/ using document root <none> and password file <none>
I20260809 08:52:21.893841 14789 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:21.893880 14789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:21.893970 14789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:21.894968 14789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "c37dd603905a499fb1483f683e73fd11"
format_stamp: "Formatted at 2026-08-09 08:52:21 on dist-test-slave-t3q3"
server_key: "ca701c9e9fd4e4516ce983790f93eb23"
server_key_iv: "ab796c6176cfef6c5b2d2dd963dbd8f6"
server_key_version: "encryptionkey@0"
I20260809 08:52:21.895289 14789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "c37dd603905a499fb1483f683e73fd11"
format_stamp: "Formatted at 2026-08-09 08:52:21 on dist-test-slave-t3q3"
server_key: "ca701c9e9fd4e4516ce983790f93eb23"
server_key_iv: "ab796c6176cfef6c5b2d2dd963dbd8f6"
server_key_version: "encryptionkey@0"
I20260809 08:52:21.896682 14789 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:52:21.897444 14810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:21.897672 14789 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:21.897754 14789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "c37dd603905a499fb1483f683e73fd11"
format_stamp: "Formatted at 2026-08-09 08:52:21 on dist-test-slave-t3q3"
server_key: "ca701c9e9fd4e4516ce983790f93eb23"
server_key_iv: "ab796c6176cfef6c5b2d2dd963dbd8f6"
server_key_version: "encryptionkey@0"
I20260809 08:52:21.897809 14789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:21.914769 14789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:21.916814 14789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:21.916930 14789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:21.920965 14789 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:35589
I20260809 08:52:21.921022 14896 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:35589 every 8 connection(s)
I20260809 08:52:21.921666 14789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:52:21.922452 14897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:21.924798 14897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c37dd603905a499fb1483f683e73fd11: Bootstrap starting.
I20260809 08:52:21.925412 14897 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c37dd603905a499fb1483f683e73fd11: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:21.925729 14897 log.cc:824] T 00000000000000000000000000000000 P c37dd603905a499fb1483f683e73fd11: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:21.926474 14897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c37dd603905a499fb1483f683e73fd11: No bootstrap required, opened a new log
I20260809 08:52:21.928128 14897 raft_consensus.cc:348] T 00000000000000000000000000000000 P c37dd603905a499fb1483f683e73fd11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c37dd603905a499fb1483f683e73fd11" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 35589 } }
I20260809 08:52:21.928294 14897 raft_consensus.cc:374] T 00000000000000000000000000000000 P c37dd603905a499fb1483f683e73fd11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:21.928344 14897 raft_consensus.cc:729] T 00000000000000000000000000000000 P c37dd603905a499fb1483f683e73fd11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c37dd603905a499fb1483f683e73fd11, State: Initialized, Role: FOLLOWER
I20260809 08:52:21.928536 14897 consensus_queue.cc:260] T 00000000000000000000000000000000 P c37dd603905a499fb1483f683e73fd11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c37dd603905a499fb1483f683e73fd11" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 35589 } }
I20260809 08:52:21.928642 14897 raft_consensus.cc:388] T 00000000000000000000000000000000 P c37dd603905a499fb1483f683e73fd11 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:52:21.928689 14897 raft_consensus.cc:482] T 00000000000000000000000000000000 P c37dd603905a499fb1483f683e73fd11 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:52:21.928750 14897 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c37dd603905a499fb1483f683e73fd11 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:21.929684 14897 raft_consensus.cc:504] T 00000000000000000000000000000000 P c37dd603905a499fb1483f683e73fd11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c37dd603905a499fb1483f683e73fd11" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 35589 } }
I20260809 08:52:21.929822 14897 leader_election.cc:302] T 00000000000000000000000000000000 P c37dd603905a499fb1483f683e73fd11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c37dd603905a499fb1483f683e73fd11; no voters:
I20260809 08:52:21.930028 14897 leader_election.cc:288] T 00000000000000000000000000000000 P c37dd603905a499fb1483f683e73fd11 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:52:21.930091 14903 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c37dd603905a499fb1483f683e73fd11 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:52:21.930297 14897 sys_catalog.cc:564] T 00000000000000000000000000000000 P c37dd603905a499fb1483f683e73fd11 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:52:21.930348 14903 raft_consensus.cc:686] T 00000000000000000000000000000000 P c37dd603905a499fb1483f683e73fd11 [term 1 LEADER]: Becoming Leader. State: Replica: c37dd603905a499fb1483f683e73fd11, State: Running, Role: LEADER
I20260809 08:52:21.930505 14903 consensus_queue.cc:237] T 00000000000000000000000000000000 P c37dd603905a499fb1483f683e73fd11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c37dd603905a499fb1483f683e73fd11" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 35589 } }
I20260809 08:52:21.930970 14904 sys_catalog.cc:455] T 00000000000000000000000000000000 P c37dd603905a499fb1483f683e73fd11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c37dd603905a499fb1483f683e73fd11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c37dd603905a499fb1483f683e73fd11" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 35589 } } }
I20260809 08:52:21.931052 14905 sys_catalog.cc:455] T 00000000000000000000000000000000 P c37dd603905a499fb1483f683e73fd11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c37dd603905a499fb1483f683e73fd11. Latest consensus state: current_term: 1 leader_uuid: "c37dd603905a499fb1483f683e73fd11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c37dd603905a499fb1483f683e73fd11" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 35589 } } }
I20260809 08:52:21.931167 14905 sys_catalog.cc:458] T 00000000000000000000000000000000 P c37dd603905a499fb1483f683e73fd11 [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:21.931068 14904 sys_catalog.cc:458] T 00000000000000000000000000000000 P c37dd603905a499fb1483f683e73fd11 [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:21.931612 14918 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:52:21.931805 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 14789
I20260809 08:52:21.931915 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:52:21.932159 8808 external_mini_cluster.cc:1428] Setting key e05a36b4b5fece7b46c3a95325b9c109
I20260809 08:52:21.932272 14918 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:52:21.934173 14918 catalog_manager.cc:1347] Generated new cluster ID: fe5eb64046364f3b86b565d03d52c03f
I20260809 08:52:21.934238 14918 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 08:52:21 dist-test-slave-t3q3 krb5kdc[14765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265541, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:21.944692 14934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:21.932789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46744 (local address 127.8.154.62:35589)
0809 08:52:21.933008 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:21.933019 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:21.933059 (+ 40us) server_negotiation.cc:407] Connection header received
0809 08:52:21.933174 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:21.933181 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:21.933228 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:21.933378 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:21.934195 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:21.935767 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:21.936918 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:21.937640 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:21.937749 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:21.940773 (+ 3024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:21.940777 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:21.940777 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:21.940790 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:21.942834 (+ 2044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:21.943276 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:21.943280 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:21.943280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:21.943396 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:21.943617 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:21.943618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:21.943618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:21.943933 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:21.944125 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:21.944274 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 08:52:21.944428 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":82,"threads_started":1}
I20260809 08:52:21.961675 14918 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:52:21.962289 14918 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:52:21.964818 14918 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c37dd603905a499fb1483f683e73fd11: Generated new TSK 0
I20260809 08:52:21.965019 14918 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab.
Entry for principal kudu/127.8.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab.
I20260809 08:52:21.991907 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.1:0
--local_ip_for_outbound_sockets=127.8.154.1
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:35589
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:41241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:22.077370 14942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:22.077617 14942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:22.077657 14942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:22.078804 14942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:22.078869 14942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:22.078915 14942 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.1
I20260809 08:52:22.080462 14942 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:41241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:35589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14942
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.1
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:22.080737 14942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:22.080973 14942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:22.086305 14958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:22.086354 14942 server_base.cc:1034] running on GCE node
W20260809 08:52:22.086359 14952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:22.086323 14951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:22.086719 14942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:22.086985 14942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:22.088132 14942 hybrid_clock.cc:648] HybridClock initialized: now 1786265542088124 us; error 36 us; skew 500 ppm
Aug 09 08:52:22 dist-test-slave-t3q3 krb5kdc[14765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265542, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:22.090863 14942 init.cc:377] Logged in from keytab as kudu/127.8.154.1@KRBTEST.COM (short username kudu)
I20260809 08:52:22.091310 14942 webserver.cc:460] Webserver started at http://127.8.154.1:43851/ using document root <none> and password file <none>
I20260809 08:52:22.091498 14942 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:22.091542 14942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:22.091634 14942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:22.092696 14942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-0/data/instance:
uuid: "a1a8b2911244495b89129545c7d91261"
format_stamp: "Formatted at 2026-08-09 08:52:22 on dist-test-slave-t3q3"
server_key: "f0260acf08266a7fcb6ea7feb443f001"
server_key_iv: "d9cc8f7c4f9d5a19eb5aacbde1f88886"
server_key_version: "encryptionkey@0"
I20260809 08:52:22.093032 14942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance:
uuid: "a1a8b2911244495b89129545c7d91261"
format_stamp: "Formatted at 2026-08-09 08:52:22 on dist-test-slave-t3q3"
server_key: "f0260acf08266a7fcb6ea7feb443f001"
server_key_iv: "d9cc8f7c4f9d5a19eb5aacbde1f88886"
server_key_version: "encryptionkey@0"
I20260809 08:52:22.094381 14942 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:52:22.095062 14971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:22.095264 14942 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:52:22.095350 14942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-0/wal
uuid: "a1a8b2911244495b89129545c7d91261"
format_stamp: "Formatted at 2026-08-09 08:52:22 on dist-test-slave-t3q3"
server_key: "f0260acf08266a7fcb6ea7feb443f001"
server_key_iv: "d9cc8f7c4f9d5a19eb5aacbde1f88886"
server_key_version: "encryptionkey@0"
I20260809 08:52:22.095404 14942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:22.109033 14942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:22.110917 14942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:22.111023 14942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:22.111196 14942 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:22.111469 14942 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:22.111500 14942 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:22.111521 14942 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:22.111533 14942 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:22.117458 14942 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.1:45619
I20260809 08:52:22.117559 15168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.1:45619 every 8 connection(s)
I20260809 08:52:22.117859 14942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
Aug 09 08:52:22 dist-test-slave-t3q3 krb5kdc[14765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265542, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:22.127141 14934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:22.118646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:45145 (local address 127.8.154.62:35589)
0809 08:52:22.118714 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:22.118717 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:22.118953 (+ 236us) server_negotiation.cc:407] Connection header received
0809 08:52:22.119949 (+ 996us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:22.119950 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:22.119965 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:22.120008 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:22.120637 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:22.121132 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:22.122184 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:22.122385 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:22.122437 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:22.124569 (+ 2132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:22.124571 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:22.124571 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:22.124578 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:22.125863 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:22.126232 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:22.126232 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:22.126233 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:22.126290 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:22.126527 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:22.126528 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:22.126528 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:22.126639 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:22.126696 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:22.126964 (+ 268us) server_negotiation.cc:299] Negotiation successful
0809 08:52:22.127075 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 08:52:22.127151 15173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:22.118615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:35589 (local address 127.8.154.1:45145)
0809 08:52:22.118824 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:22.118863 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:22.119841 (+ 978us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:22.120092 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:22.120096 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:22.120359 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:22.120559 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:22.120563 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:22.121225 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:22.121226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:22.122107 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:22.122110 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:22.122468 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:22.122469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:22.122579 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:22.123032 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:22.123041 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:22.124472 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:22.125938 (+ 1466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:22.125944 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:22.125946 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:22.126161 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:22.126353 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:22.126353 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:22.126354 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:22.126452 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:22.126698 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:22.126702 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:22.126808 (+ 106us) client_negotiation.cc:769] Sending connection context
0809 08:52:22.126888 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 08:52:22.127001 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":110,"threads_started":1}
I20260809 08:52:22.127885 15170 heartbeater.cc:347] Connected to a master server at 127.8.154.62:35589
I20260809 08:52:22.127961 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 14942
I20260809 08:52:22.127979 15170 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:22.128086 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance
I20260809 08:52:22.128284 8808 external_mini_cluster.cc:1428] Setting key da0c20e5220c4055e1448dd49e69da2b
I20260809 08:52:22.129240 15170 heartbeater.cc:511] Master 127.8.154.62:35589 requested a full tablet report, sending...
I20260809 08:52:22.129943 14838 ts_manager.cc:194] Registered new tserver with Master: a1a8b2911244495b89129545c7d91261 (127.8.154.1:45619)
I20260809 08:52:22.130841 14838 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.1@KRBTEST.COM'} at 127.8.154.1:45145
WARNING: no policy specified for kudu/127.8.154.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab.
Entry for principal kudu/127.8.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab.
I20260809 08:52:22.156833 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.2:0
--local_ip_for_outbound_sockets=127.8.154.2
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:35589
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:41241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:22.243459 15176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:22.243692 15176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:22.243732 15176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:22.244905 15176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:22.244974 15176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:22.245020 15176 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.2
I20260809 08:52:22.246547 15176 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:41241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:35589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.2
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:22.246815 15176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:22.247056 15176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:22.252101 15185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:22.252122 15183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:22.252110 15188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:22.252641 15176 server_base.cc:1034] running on GCE node
I20260809 08:52:22.252876 15176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:22.253154 15176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:22.254315 15176 hybrid_clock.cc:648] HybridClock initialized: now 1786265542254301 us; error 32 us; skew 500 ppm
Aug 09 08:52:22 dist-test-slave-t3q3 krb5kdc[14765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265542, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:22.256991 15176 init.cc:377] Logged in from keytab as kudu/127.8.154.2@KRBTEST.COM (short username kudu)
I20260809 08:52:22.257412 15176 webserver.cc:460] Webserver started at http://127.8.154.2:40465/ using document root <none> and password file <none>
I20260809 08:52:22.257637 15176 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:22.257704 15176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:22.257860 15176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:22.258888 15176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-1/data/instance:
uuid: "213b9592b6d04bf7871ef9e4f153b914"
format_stamp: "Formatted at 2026-08-09 08:52:22 on dist-test-slave-t3q3"
server_key: "bf6d067d846fe72c81d609e0942b0315"
server_key_iv: "0616ecf9f7a7f627a575150ab86f127c"
server_key_version: "encryptionkey@0"
I20260809 08:52:22.259250 15176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance:
uuid: "213b9592b6d04bf7871ef9e4f153b914"
format_stamp: "Formatted at 2026-08-09 08:52:22 on dist-test-slave-t3q3"
server_key: "bf6d067d846fe72c81d609e0942b0315"
server_key_iv: "0616ecf9f7a7f627a575150ab86f127c"
server_key_version: "encryptionkey@0"
I20260809 08:52:22.260689 15176 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260809 08:52:22.261415 15202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:22.261595 15176 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:52:22.261703 15176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-1/wal
uuid: "213b9592b6d04bf7871ef9e4f153b914"
format_stamp: "Formatted at 2026-08-09 08:52:22 on dist-test-slave-t3q3"
server_key: "bf6d067d846fe72c81d609e0942b0315"
server_key_iv: "0616ecf9f7a7f627a575150ab86f127c"
server_key_version: "encryptionkey@0"
I20260809 08:52:22.261775 15176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:22.307958 15176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:22.310379 15176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:22.310554 15176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:22.310812 15176 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:22.311133 15176 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:22.311189 15176 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:22.311234 15176 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:22.311259 15176 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:22.317317 15176 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.2:35621
I20260809 08:52:22.317365 15375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.2:35621 every 8 connection(s)
I20260809 08:52:22.317700 15176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
I20260809 08:52:22.322947 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 15176
I20260809 08:52:22.323108 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance
I20260809 08:52:22.323307 8808 external_mini_cluster.cc:1428] Setting key 95472c57ae45cd06abfc23cabe01293f
Aug 09 08:52:22 dist-test-slave-t3q3 krb5kdc[14765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265542, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:22.327487 15382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:22.318411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:35589 (local address 127.8.154.2:45201)
0809 08:52:22.318602 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:22.318644 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:22.319590 (+ 946us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:22.319819 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:22.319822 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:22.320153 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:22.320361 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:22.320366 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:22.321045 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:22.321046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:22.321843 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:22.321845 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:22.322188 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:22.322189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:22.322289 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:22.322736 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:22.322744 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:22.324470 (+ 1726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:22.326124 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:22.326130 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:22.326132 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:22.326353 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:22.326588 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:22.326588 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:22.326589 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:22.326687 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:22.326992 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:22.326998 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:22.327125 (+ 127us) client_negotiation.cc:769] Sending connection context
0809 08:52:22.327213 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 08:52:22.327331 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":111,"threads_started":1}
I20260809 08:52:22.327497 14934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:22.318429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:45201 (local address 127.8.154.62:35589)
0809 08:52:22.318497 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:22.318499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:22.318722 (+ 223us) server_negotiation.cc:407] Connection header received
0809 08:52:22.319689 (+ 967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:22.319690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:22.319701 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:22.319764 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:22.320437 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:22.320938 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:22.321917 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:22.322115 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:22.322167 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:22.324574 (+ 2407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:22.324576 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:22.324576 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:22.324582 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:22.326029 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:22.326428 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:22.326429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:22.326430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:22.326497 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:22.326754 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:22.326755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:22.326755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:22.326897 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:22.326982 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:22.327263 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 08:52:22.327358 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 08:52:22.328117 15378 heartbeater.cc:347] Connected to a master server at 127.8.154.62:35589
I20260809 08:52:22.328235 15378 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:22.328411 15378 heartbeater.cc:511] Master 127.8.154.62:35589 requested a full tablet report, sending...
I20260809 08:52:22.328856 14838 ts_manager.cc:194] Registered new tserver with Master: 213b9592b6d04bf7871ef9e4f153b914 (127.8.154.2:35621)
I20260809 08:52:22.329402 14838 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.2@KRBTEST.COM'} at 127.8.154.2:45201
WARNING: no policy specified for kudu/127.8.154.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab.
Entry for principal kudu/127.8.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab.
I20260809 08:52:22.348815 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.3:0
--local_ip_for_outbound_sockets=127.8.154.3
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:35589
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:41241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:22.431620 15390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:22.431861 15390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:22.431885 15390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:22.433118 15390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:22.433166 15390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:22.433192 15390 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.3
I20260809 08:52:22.434684 15390 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:41241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:35589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.3
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:22.434922 15390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:22.435176 15390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 08:52:22.440410 15390 server_base.cc:1034] running on GCE node
W20260809 08:52:22.440362 15398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:22.440375 15400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:22.440411 15397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:22.440866 15390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:22.441128 15390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:22.442294 15390 hybrid_clock.cc:648] HybridClock initialized: now 1786265542442292 us; error 42 us; skew 500 ppm
Aug 09 08:52:22 dist-test-slave-t3q3 krb5kdc[14765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265542, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:22.444983 15390 init.cc:377] Logged in from keytab as kudu/127.8.154.3@KRBTEST.COM (short username kudu)
I20260809 08:52:22.445446 15390 webserver.cc:460] Webserver started at http://127.8.154.3:45447/ using document root <none> and password file <none>
I20260809 08:52:22.445676 15390 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:22.445724 15390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:22.445879 15390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:22.446933 15390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-2/data/instance:
uuid: "bdcfd5e33601427bae0e122988112b31"
format_stamp: "Formatted at 2026-08-09 08:52:22 on dist-test-slave-t3q3"
server_key: "76fa3580de262108112f33700380cee2"
server_key_iv: "233e210f8070c6c3c9275d45746926dd"
server_key_version: "encryptionkey@0"
I20260809 08:52:22.447307 15390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance:
uuid: "bdcfd5e33601427bae0e122988112b31"
format_stamp: "Formatted at 2026-08-09 08:52:22 on dist-test-slave-t3q3"
server_key: "76fa3580de262108112f33700380cee2"
server_key_iv: "233e210f8070c6c3c9275d45746926dd"
server_key_version: "encryptionkey@0"
I20260809 08:52:22.448745 15390 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:52:22.449477 15410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:22.449694 15390 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:22.449802 15390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-2/wal
uuid: "bdcfd5e33601427bae0e122988112b31"
format_stamp: "Formatted at 2026-08-09 08:52:22 on dist-test-slave-t3q3"
server_key: "76fa3580de262108112f33700380cee2"
server_key_iv: "233e210f8070c6c3c9275d45746926dd"
server_key_version: "encryptionkey@0"
I20260809 08:52:22.449885 15390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:22.459992 15390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:22.461918 15390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:22.462090 15390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:22.462355 15390 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:22.462666 15390 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:22.462723 15390 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:22.462764 15390 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:22.462791 15390 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:22.469060 15390 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.3:45207
I20260809 08:52:22.469131 15602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.3:45207 every 8 connection(s)
I20260809 08:52:22.469444 15390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
I20260809 08:52:22.473809 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 15390
I20260809 08:52:22.473909 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance
I20260809 08:52:22.474120 8808 external_mini_cluster.cc:1428] Setting key 5cd01faaf40c0b223b05195a29aae4c8
Aug 09 08:52:22 dist-test-slave-t3q3 krb5kdc[14765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265542, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:22.478547 14934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:22.470155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:55431 (local address 127.8.154.62:35589)
0809 08:52:22.470227 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:22.470229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:22.470435 (+ 206us) server_negotiation.cc:407] Connection header received
0809 08:52:22.471298 (+ 863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:22.471299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:22.471312 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:22.471379 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:22.471974 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:22.472558 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:22.473582 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:22.473763 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:22.473814 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:22.475998 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:22.475999 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:22.475999 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:22.476006 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:22.477430 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:22.477790 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:22.477791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:22.477791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:22.477843 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:22.478054 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:22.478055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:22.478055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:22.478153 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:22.478203 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:22.478426 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 08:52:22.478486 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 08:52:22.478717 15607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:22.470118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:35589 (local address 127.8.154.3:55431)
0809 08:52:22.470285 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:22.470327 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:22.471195 (+ 868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:22.471442 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:22.471445 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:22.471697 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:22.471881 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:22.471885 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:22.472645 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:22.472646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:22.473499 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:22.473502 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:22.473849 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:22.473850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:22.474003 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:22.474553 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:22.474561 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:22.475911 (+ 1350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:22.477501 (+ 1590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:22.477513 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:22.477515 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:22.477724 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:22.477905 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:22.477906 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:22.477906 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:22.477997 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:22.478208 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:22.478212 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:22.478317 (+ 105us) client_negotiation.cc:769] Sending connection context
0809 08:52:22.478396 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 08:52:22.478520 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
I20260809 08:52:22.479218 15604 heartbeater.cc:347] Connected to a master server at 127.8.154.62:35589
I20260809 08:52:22.479328 15604 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:22.479506 15604 heartbeater.cc:511] Master 127.8.154.62:35589 requested a full tablet report, sending...
I20260809 08:52:22.479861 14838 ts_manager.cc:194] Registered new tserver with Master: bdcfd5e33601427bae0e122988112b31 (127.8.154.3:45207)
I20260809 08:52:22.480293 14838 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.3@KRBTEST.COM'} at 127.8.154.3:55431
I20260809 08:52:22.485689 8808 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 08:52:22 dist-test-slave-t3q3 krb5kdc[14765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 08:52:22 dist-test-slave-t3q3 krb5kdc[14765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.154.1@KRBTEST.COM
I20260809 08:52:22.509228 15613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:22.501858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50658 (local address 127.8.154.1:45619)
0809 08:52:22.502026 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:22.502031 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:22.502041 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:52:22.502066 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:22.502068 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:22.502082 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:22.502147 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:22.502699 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:22.503391 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:22.504244 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:22.504553 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:22.504611 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:22.506679 (+ 2068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:22.506687 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:22.506687 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:22.506693 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:22.508162 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:22.508463 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:22.508466 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:22.508466 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:22.508521 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:22.508686 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:22.508687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:22.508687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:22.508821 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:22.508932 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:22.509058 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 08:52:22.509120 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":90,"threads_started":1}
Aug 09 08:52:22 dist-test-slave-t3q3 krb5kdc[14765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:22.516952 14934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:22.510606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46746 (local address 127.8.154.62:35589)
0809 08:52:22.510672 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:22.510674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:22.510681 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:52:22.510738 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:22.510739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:22.510748 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:22.510786 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:22.511234 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:22.511692 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:22.512443 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:22.512625 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:22.512678 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:22.514713 (+ 2035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:22.514714 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:22.514715 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:22.514721 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:22.515846 (+ 1125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:22.516298 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:22.516299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:22.516299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:22.516351 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:22.516514 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:22.516515 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:22.516515 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:22.516639 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:22.516710 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:22.516792 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 08:52:22.516850 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260809 08:52:22.517189 14838 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46746
I20260809 08:52:22.517503 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 14942
I20260809 08:52:22.536470 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 15176
I20260809 08:52:22.542271 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 15390
I20260809 08:52:22.556368 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 14789
2026-08-09T08:52:22Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2148 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:52:22 dist-test-slave-t3q3 krb5kdc[15621](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:52:22 dist-test-slave-t3q3 krb5kdc[15621](info): set up 2 sockets
Aug 09 08:52:22 dist-test-slave-t3q3 krb5kdc[15621](info): commencing operation
krb5kdc: starting...
W20260809 08:52:23.807965 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.227s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:52:23 dist-test-slave-t3q3 krb5kdc[15621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265543, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:52:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:52:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
I20260809 08:52:23.939358 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:32863
--webserver_interface=127.8.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:46655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:32863
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:24.021446 15646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:24.021667 15646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:24.021704 15646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:24.022835 15646 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:52:24.022899 15646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:24.022924 15646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:24.022955 15646 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:52:24.022979 15646 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:52:24.024516 15646 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:46655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:32863
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:32863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:24.024775 15646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:24.024996 15646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:24.029933 15658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:24.029911 15653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:24.029939 15646 server_base.cc:1034] running on GCE node
W20260809 08:52:24.029892 15655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:24.030412 15646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:24.030715 15646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:24.031913 15646 hybrid_clock.cc:648] HybridClock initialized: now 1786265544031886 us; error 36 us; skew 500 ppm
Aug 09 08:52:24 dist-test-slave-t3q3 krb5kdc[15621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265544, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:24.038040 15646 init.cc:377] Logged in from keytab as kudu/127.8.154.62@KRBTEST.COM (short username kudu)
I20260809 08:52:24.038359 15646 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:24.038404 15646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:24.038493 15646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:24.039500 15646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "440ec2dd16514576b5dd07fa590ba5cc"
format_stamp: "Formatted at 2026-08-09 08:52:24 on dist-test-slave-t3q3"
server_key: "bfcd6dece3b33edebbbf2f514e420ae5"
server_key_iv: "237383a5bd7301b0642993313fc9008b"
server_key_version: "encryptionkey@0"
I20260809 08:52:24.039822 15646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "440ec2dd16514576b5dd07fa590ba5cc"
format_stamp: "Formatted at 2026-08-09 08:52:24 on dist-test-slave-t3q3"
server_key: "bfcd6dece3b33edebbbf2f514e420ae5"
server_key_iv: "237383a5bd7301b0642993313fc9008b"
server_key_version: "encryptionkey@0"
I20260809 08:52:24.041239 15646 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 08:52:24.041935 15665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:24.042122 15646 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:24.042215 15646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "440ec2dd16514576b5dd07fa590ba5cc"
format_stamp: "Formatted at 2026-08-09 08:52:24 on dist-test-slave-t3q3"
server_key: "bfcd6dece3b33edebbbf2f514e420ae5"
server_key_iv: "237383a5bd7301b0642993313fc9008b"
server_key_version: "encryptionkey@0"
I20260809 08:52:24.042270 15646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:24.057535 15646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:24.059460 15646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:24.059573 15646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:24.063427 15646 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:32863
I20260809 08:52:24.063489 15753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:32863 every 8 connection(s)
I20260809 08:52:24.063767 15646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:52:24.064749 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 15646
I20260809 08:52:24.064872 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:52:24.065012 15754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:24.065059 8808 external_mini_cluster.cc:1428] Setting key 95e747c6c99914f49195057b646820cf
I20260809 08:52:24.068447 15754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 440ec2dd16514576b5dd07fa590ba5cc: Bootstrap starting.
I20260809 08:52:24.069092 15754 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 440ec2dd16514576b5dd07fa590ba5cc: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:24.069382 15754 log.cc:824] T 00000000000000000000000000000000 P 440ec2dd16514576b5dd07fa590ba5cc: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:24.070106 15754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 440ec2dd16514576b5dd07fa590ba5cc: No bootstrap required, opened a new log
Aug 09 08:52:24 dist-test-slave-t3q3 krb5kdc[15621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265543, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:24.072650 15754 raft_consensus.cc:348] T 00000000000000000000000000000000 P 440ec2dd16514576b5dd07fa590ba5cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "440ec2dd16514576b5dd07fa590ba5cc" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 32863 } }
I20260809 08:52:24.072868 15754 raft_consensus.cc:374] T 00000000000000000000000000000000 P 440ec2dd16514576b5dd07fa590ba5cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:24.072914 15754 raft_consensus.cc:729] T 00000000000000000000000000000000 P 440ec2dd16514576b5dd07fa590ba5cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 440ec2dd16514576b5dd07fa590ba5cc, State: Initialized, Role: FOLLOWER
I20260809 08:52:24.073065 15754 consensus_queue.cc:260] T 00000000000000000000000000000000 P 440ec2dd16514576b5dd07fa590ba5cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "440ec2dd16514576b5dd07fa590ba5cc" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 32863 } }
I20260809 08:52:24.073154 15754 raft_consensus.cc:388] T 00000000000000000000000000000000 P 440ec2dd16514576b5dd07fa590ba5cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:52:24.073200 15754 raft_consensus.cc:482] T 00000000000000000000000000000000 P 440ec2dd16514576b5dd07fa590ba5cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:52:24.073240 15754 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 440ec2dd16514576b5dd07fa590ba5cc [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:24.073820 15754 raft_consensus.cc:504] T 00000000000000000000000000000000 P 440ec2dd16514576b5dd07fa590ba5cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "440ec2dd16514576b5dd07fa590ba5cc" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 32863 } }
I20260809 08:52:24.073911 15754 leader_election.cc:302] T 00000000000000000000000000000000 P 440ec2dd16514576b5dd07fa590ba5cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 440ec2dd16514576b5dd07fa590ba5cc; no voters:
I20260809 08:52:24.074116 15754 leader_election.cc:288] T 00000000000000000000000000000000 P 440ec2dd16514576b5dd07fa590ba5cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:52:24.074167 15762 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 440ec2dd16514576b5dd07fa590ba5cc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:52:24.074333 15762 raft_consensus.cc:686] T 00000000000000000000000000000000 P 440ec2dd16514576b5dd07fa590ba5cc [term 1 LEADER]: Becoming Leader. State: Replica: 440ec2dd16514576b5dd07fa590ba5cc, State: Running, Role: LEADER
I20260809 08:52:24.074404 15754 sys_catalog.cc:564] T 00000000000000000000000000000000 P 440ec2dd16514576b5dd07fa590ba5cc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:52:24.074429 15762 consensus_queue.cc:237] T 00000000000000000000000000000000 P 440ec2dd16514576b5dd07fa590ba5cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "440ec2dd16514576b5dd07fa590ba5cc" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 32863 } }
I20260809 08:52:24.074762 15762 sys_catalog.cc:455] T 00000000000000000000000000000000 P 440ec2dd16514576b5dd07fa590ba5cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 440ec2dd16514576b5dd07fa590ba5cc. Latest consensus state: current_term: 1 leader_uuid: "440ec2dd16514576b5dd07fa590ba5cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "440ec2dd16514576b5dd07fa590ba5cc" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 32863 } } }
I20260809 08:52:24.074851 15762 sys_catalog.cc:458] T 00000000000000000000000000000000 P 440ec2dd16514576b5dd07fa590ba5cc [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:24.074788 15764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 440ec2dd16514576b5dd07fa590ba5cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "440ec2dd16514576b5dd07fa590ba5cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "440ec2dd16514576b5dd07fa590ba5cc" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 32863 } } }
I20260809 08:52:24.075042 15764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 440ec2dd16514576b5dd07fa590ba5cc [sys.catalog]: This master's current role is: LEADER
W20260809 08:52:24.076485 15789 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 440ec2dd16514576b5dd07fa590ba5cc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260809 08:52:24.076514 15758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:24.065608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35992 (local address 127.8.154.62:32863)
0809 08:52:24.065797 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:24.065807 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:24.065840 (+ 33us) server_negotiation.cc:407] Connection header received
0809 08:52:24.066299 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:24.066308 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:24.066372 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:24.066526 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:24.067112 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:24.068502 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:24.069322 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:24.069773 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:24.069890 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:24.072473 (+ 2583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:24.072477 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:24.072478 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:24.072490 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:24.074729 (+ 2239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:24.075185 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:24.075188 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:24.075189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:24.075282 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:24.075533 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:24.075534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:24.075535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:24.075875 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:24.076038 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:24.076138 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 08:52:24.076295 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":82,"threads_started":1}
W20260809 08:52:24.076561 15789 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:52:24.076622 15771 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:52:24.076768 15771 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:52:24.078321 15771 catalog_manager.cc:1347] Generated new cluster ID: 47d952bfbc644e9588872de4067bd321
I20260809 08:52:24.078382 15771 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:52:24.087888 15771 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:52:24.088474 15771 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:52:24.096865 15771 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 440ec2dd16514576b5dd07fa590ba5cc: Generated new TSK 0
I20260809 08:52:24.097047 15771 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab.
Entry for principal kudu/127.8.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab.
I20260809 08:52:24.131673 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.1:0
--local_ip_for_outbound_sockets=127.8.154.1
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:32863
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:46655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:24.229173 15797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:24.229404 15797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:24.229449 15797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:24.230623 15797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:24.230687 15797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:24.230734 15797 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.1
I20260809 08:52:24.232355 15797 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:46655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:32863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15797
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.1
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:24.232620 15797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:24.232853 15797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:24.237993 15804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:24.237991 15806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:24.237991 15803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:24.238279 15797 server_base.cc:1034] running on GCE node
I20260809 08:52:24.238523 15797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:24.238811 15797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:24.239964 15797 hybrid_clock.cc:648] HybridClock initialized: now 1786265544239940 us; error 36 us; skew 500 ppm
Aug 09 08:52:24 dist-test-slave-t3q3 krb5kdc[15621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265544, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:24.242470 15797 init.cc:377] Logged in from keytab as kudu/127.8.154.1@KRBTEST.COM (short username kudu)
I20260809 08:52:24.242820 15797 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:24.242892 15797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:24.243052 15797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:24.244174 15797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-0/data/instance:
uuid: "bd832ae748a64b6683993285708076ea"
format_stamp: "Formatted at 2026-08-09 08:52:24 on dist-test-slave-t3q3"
server_key: "954b39c849d7ce3133d0ca1913ea796f"
server_key_iv: "cee6fc9d7282681cb6d773ddaa99b389"
server_key_version: "encryptionkey@0"
I20260809 08:52:24.244542 15797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance:
uuid: "bd832ae748a64b6683993285708076ea"
format_stamp: "Formatted at 2026-08-09 08:52:24 on dist-test-slave-t3q3"
server_key: "954b39c849d7ce3133d0ca1913ea796f"
server_key_iv: "cee6fc9d7282681cb6d773ddaa99b389"
server_key_version: "encryptionkey@0"
I20260809 08:52:24.245954 15797 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 08:52:24.246730 15816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:24.246958 15797 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:52:24.247083 15797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-0/wal
uuid: "bd832ae748a64b6683993285708076ea"
format_stamp: "Formatted at 2026-08-09 08:52:24 on dist-test-slave-t3q3"
server_key: "954b39c849d7ce3133d0ca1913ea796f"
server_key_iv: "cee6fc9d7282681cb6d773ddaa99b389"
server_key_version: "encryptionkey@0"
I20260809 08:52:24.247159 15797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:24.288851 15797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:24.290865 15797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:24.291046 15797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:24.291308 15797 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:24.291653 15797 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:24.291710 15797 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:24.291746 15797 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:24.291795 15797 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:24.297752 15797 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.1:43125
I20260809 08:52:24.297802 16026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.1:43125 every 8 connection(s)
I20260809 08:52:24.298090 15797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
I20260809 08:52:24.298364 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 15797
I20260809 08:52:24.298486 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance
I20260809 08:52:24.298719 8808 external_mini_cluster.cc:1428] Setting key bf6113e263fde41b19fae03339c05345
Aug 09 08:52:24 dist-test-slave-t3q3 krb5kdc[15621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265544, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:24.309093 15758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:24.299021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:56551 (local address 127.8.154.62:32863)
0809 08:52:24.299073 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:24.299075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:24.299485 (+ 410us) server_negotiation.cc:407] Connection header received
0809 08:52:24.300282 (+ 797us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:24.300283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:24.300293 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:24.300342 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:24.301702 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:24.302218 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:24.303277 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:24.303606 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:24.303682 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:24.305912 (+ 2230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:24.305914 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:24.305915 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:24.305925 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:24.307606 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:24.308025 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:24.308027 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:24.308028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:24.308128 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:24.308386 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:24.308387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:24.308388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:24.308543 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:24.308620 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:24.308883 (+ 263us) server_negotiation.cc:299] Negotiation successful
0809 08:52:24.308985 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 08:52:24.309141 16030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:24.299037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:32863 (local address 127.8.154.1:56551)
0809 08:52:24.299251 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:24.299290 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:24.300161 (+ 871us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:24.300427 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:24.300432 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:24.300830 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:24.301594 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:24.301601 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:24.302321 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:24.302323 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:24.303151 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:24.303153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:24.303705 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:24.303706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:24.303835 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:24.304388 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:24.304396 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:24.305809 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:24.307704 (+ 1895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:24.307711 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:24.307712 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:24.307938 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:24.308204 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:24.308206 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:24.308206 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:24.308313 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:24.308616 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:24.308621 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:24.308740 (+ 119us) client_negotiation.cc:769] Sending connection context
0809 08:52:24.308826 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 08:52:24.308951 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":121,"threads_started":1}
WARNING: no policy specified for kudu/127.8.154.2@KRBTEST.COM; defaulting to no policy
I20260809 08:52:24.309782 16028 heartbeater.cc:347] Connected to a master server at 127.8.154.62:32863
I20260809 08:52:24.309888 16028 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:24.310065 16028 heartbeater.cc:511] Master 127.8.154.62:32863 requested a full tablet report, sending...
I20260809 08:52:24.310654 15691 ts_manager.cc:194] Registered new tserver with Master: bd832ae748a64b6683993285708076ea (127.8.154.1:43125)
I20260809 08:52:24.311331 15691 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.1@KRBTEST.COM'} at 127.8.154.1:56551
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab.
Entry for principal kudu/127.8.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab.
I20260809 08:52:24.326792 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.2:0
--local_ip_for_outbound_sockets=127.8.154.2
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:32863
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:46655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:24.409473 16035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:24.409690 16035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:24.409714 16035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:24.411031 16035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:24.411075 16035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:24.411103 16035 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.2
I20260809 08:52:24.412730 16035 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:46655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:32863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.2
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:24.412999 16035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:24.413240 16035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:24.418376 16047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:24.418371 16044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:24.418367 16043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:24.418848 16035 server_base.cc:1034] running on GCE node
I20260809 08:52:24.419088 16035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:24.419358 16035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:24.420540 16035 hybrid_clock.cc:648] HybridClock initialized: now 1786265544420578 us; error 101 us; skew 500 ppm
Aug 09 08:52:24 dist-test-slave-t3q3 krb5kdc[15621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265544, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:24.423079 16035 init.cc:377] Logged in from keytab as kudu/127.8.154.2@KRBTEST.COM (short username kudu)
I20260809 08:52:24.423444 16035 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:24.423532 16035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:24.423684 16035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:24.424896 16035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-1/data/instance:
uuid: "2a6b71984e1c41c295a30720403128a7"
format_stamp: "Formatted at 2026-08-09 08:52:24 on dist-test-slave-t3q3"
server_key: "582e764d0a184986309aa7bfeab9d097"
server_key_iv: "1a91f9b8afaecab55287f4b0afa525b8"
server_key_version: "encryptionkey@0"
I20260809 08:52:24.425279 16035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance:
uuid: "2a6b71984e1c41c295a30720403128a7"
format_stamp: "Formatted at 2026-08-09 08:52:24 on dist-test-slave-t3q3"
server_key: "582e764d0a184986309aa7bfeab9d097"
server_key_iv: "1a91f9b8afaecab55287f4b0afa525b8"
server_key_version: "encryptionkey@0"
I20260809 08:52:24.426664 16035 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 08:52:24.427415 16058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:24.427608 16035 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:52:24.427732 16035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-1/wal
uuid: "2a6b71984e1c41c295a30720403128a7"
format_stamp: "Formatted at 2026-08-09 08:52:24 on dist-test-slave-t3q3"
server_key: "582e764d0a184986309aa7bfeab9d097"
server_key_iv: "1a91f9b8afaecab55287f4b0afa525b8"
server_key_version: "encryptionkey@0"
I20260809 08:52:24.427793 16035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:24.440655 16035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:24.442898 16035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:24.443027 16035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:24.443236 16035 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:24.443545 16035 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:24.443583 16035 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:24.443607 16035 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:24.443621 16035 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:24.450672 16035 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.2:39955
I20260809 08:52:24.450706 16242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.2:39955 every 8 connection(s)
I20260809 08:52:24.451081 16035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
I20260809 08:52:24.452160 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 16035
I20260809 08:52:24.452275 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance
I20260809 08:52:24.452491 8808 external_mini_cluster.cc:1428] Setting key 72045c67203263ac1ab08d95c093fabd
Aug 09 08:52:24 dist-test-slave-t3q3 krb5kdc[15621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265544, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:24.461551 15758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:24.451860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:43341 (local address 127.8.154.62:32863)
0809 08:52:24.451902 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:24.451904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:24.452244 (+ 340us) server_negotiation.cc:407] Connection header received
0809 08:52:24.452935 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:24.452937 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:24.452951 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:24.453046 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:24.454440 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:24.455026 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:24.456149 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:24.456408 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:24.456470 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:24.458644 (+ 2174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:24.458646 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:24.458646 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:24.458654 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:24.460141 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:24.460540 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:24.460541 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:24.460542 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:24.460616 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:24.460878 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:24.460879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:24.460879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:24.461003 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:24.461075 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:24.461368 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 08:52:24.461455 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 08:52:24.461751 16247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:24.451843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:32863 (local address 127.8.154.2:43341)
0809 08:52:24.452005 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:24.452077 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:24.452813 (+ 736us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:24.453108 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:24.453111 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:24.453532 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:24.454284 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:24.454291 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:24.455160 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:24.455161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:24.456005 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:24.456008 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:24.456487 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:24.456488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:24.456598 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:24.457164 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:24.457172 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:24.458537 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:24.460228 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:24.460236 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:24.460238 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:24.460461 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:24.460687 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:24.460688 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:24.460689 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:24.460802 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:24.461078 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:24.461093 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:24.461232 (+ 139us) client_negotiation.cc:769] Sending connection context
0809 08:52:24.461344 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 08:52:24.461506 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":77,"threads_started":1}
WARNING: no policy specified for kudu/127.8.154.3@KRBTEST.COM; defaulting to no policy
I20260809 08:52:24.462260 16245 heartbeater.cc:347] Connected to a master server at 127.8.154.62:32863
I20260809 08:52:24.462350 16245 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:24.462531 16245 heartbeater.cc:511] Master 127.8.154.62:32863 requested a full tablet report, sending...
I20260809 08:52:24.462972 15691 ts_manager.cc:194] Registered new tserver with Master: 2a6b71984e1c41c295a30720403128a7 (127.8.154.2:39955)
I20260809 08:52:24.463435 15691 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.2@KRBTEST.COM'} at 127.8.154.2:43341
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab.
Entry for principal kudu/127.8.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab.
I20260809 08:52:24.482524 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.3:0
--local_ip_for_outbound_sockets=127.8.154.3
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:32863
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:46655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:24.565824 16250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:24.566026 16250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:24.566051 16250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:24.567219 16250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:24.567263 16250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:24.567291 16250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.3
I20260809 08:52:24.568889 16250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:46655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:32863
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16250
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.3
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:24.569133 16250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:24.569376 16250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:24.574509 16257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:24.574625 16250 server_base.cc:1034] running on GCE node
W20260809 08:52:24.574508 16260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:24.574659 16258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:24.575008 16250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:24.575282 16250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:24.576455 16250 hybrid_clock.cc:648] HybridClock initialized: now 1786265544576451 us; error 53 us; skew 500 ppm
Aug 09 08:52:24 dist-test-slave-t3q3 krb5kdc[15621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265544, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:24.578961 16250 init.cc:377] Logged in from keytab as kudu/127.8.154.3@KRBTEST.COM (short username kudu)
I20260809 08:52:24.579304 16250 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:24.579378 16250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:24.579535 16250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:24.580619 16250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-2/data/instance:
uuid: "ef0162763bdb43d5ab385ee232fd1482"
format_stamp: "Formatted at 2026-08-09 08:52:24 on dist-test-slave-t3q3"
server_key: "435013239e2b30f306047e61554d15c0"
server_key_iv: "ba1b9b6b0e99abad3cfad0af0600847c"
server_key_version: "encryptionkey@0"
I20260809 08:52:24.581017 16250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance:
uuid: "ef0162763bdb43d5ab385ee232fd1482"
format_stamp: "Formatted at 2026-08-09 08:52:24 on dist-test-slave-t3q3"
server_key: "435013239e2b30f306047e61554d15c0"
server_key_iv: "ba1b9b6b0e99abad3cfad0af0600847c"
server_key_version: "encryptionkey@0"
I20260809 08:52:24.582376 16250 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:52:24.583086 16275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:24.583263 16250 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:52:24.583371 16250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-2/wal
uuid: "ef0162763bdb43d5ab385ee232fd1482"
format_stamp: "Formatted at 2026-08-09 08:52:24 on dist-test-slave-t3q3"
server_key: "435013239e2b30f306047e61554d15c0"
server_key_iv: "ba1b9b6b0e99abad3cfad0af0600847c"
server_key_version: "encryptionkey@0"
I20260809 08:52:24.583447 16250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:24.596119 16250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:24.597954 16250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:24.598120 16250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:24.598340 16250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:24.598649 16250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:24.598713 16250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:24.598757 16250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:24.598783 16250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:24.605273 16250 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.3:46131
I20260809 08:52:24.605324 16448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.3:46131 every 8 connection(s)
I20260809 08:52:24.605618 16250 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
I20260809 08:52:24.607964 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 16250
I20260809 08:52:24.608104 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance
I20260809 08:52:24.608311 8808 external_mini_cluster.cc:1428] Setting key 697a3909b4011ad92c2e544b7f673fea
Aug 09 08:52:24 dist-test-slave-t3q3 krb5kdc[15621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265544, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:24.614852 15758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:24.606378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:56605 (local address 127.8.154.62:32863)
0809 08:52:24.606460 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:24.606463 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:24.606645 (+ 182us) server_negotiation.cc:407] Connection header received
0809 08:52:24.607089 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:24.607090 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:24.607101 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:24.607154 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:24.608126 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:24.608918 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:24.610044 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:24.610241 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:24.610296 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:24.612355 (+ 2059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:24.612357 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:24.612357 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:24.612363 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:24.613591 (+ 1228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:24.614021 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:24.614022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:24.614023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:24.614076 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:24.614306 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:24.614306 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:24.614306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:24.614409 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:24.614461 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:24.614728 (+ 267us) server_negotiation.cc:299] Negotiation successful
0809 08:52:24.614789 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 08:52:24.615108 16454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:24.606352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:32863 (local address 127.8.154.3:56605)
0809 08:52:24.606529 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:24.606563 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:24.607009 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:24.607208 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:24.607211 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:24.607459 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:24.607971 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:24.607976 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:24.609021 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:24.609022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:24.609965 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:24.609968 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:24.610323 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:24.610324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:24.610427 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:24.610852 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:24.610861 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:24.612269 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:24.613692 (+ 1423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:24.613699 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:24.613701 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:24.613949 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:24.614139 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:24.614139 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:24.614139 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:24.614247 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:24.614467 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:24.614472 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:24.614583 (+ 111us) client_negotiation.cc:769] Sending connection context
0809 08:52:24.614708 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 08:52:24.614879 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":105,"threads_started":1}
I20260809 08:52:24.615628 16449 heartbeater.cc:347] Connected to a master server at 127.8.154.62:32863
I20260809 08:52:24.615736 16449 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:24.615916 16449 heartbeater.cc:511] Master 127.8.154.62:32863 requested a full tablet report, sending...
I20260809 08:52:24.616324 15691 ts_manager.cc:194] Registered new tserver with Master: ef0162763bdb43d5ab385ee232fd1482 (127.8.154.3:46131)
I20260809 08:52:24.616719 15691 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.3@KRBTEST.COM'} at 127.8.154.3:56605
I20260809 08:52:24.619825 8808 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:52:24.626950 15758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:24.621245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35994 (local address 127.8.154.62:32863)
0809 08:52:24.621300 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:24.621302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:24.621395 (+ 93us) server_negotiation.cc:407] Connection header received
0809 08:52:24.621440 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:24.621440 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:24.621448 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:24.621495 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:24.622014 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:24.622472 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:24.623144 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:24.623330 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:24.623379 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:24.624520 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:24.624521 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:24.624522 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:24.624528 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:24.625909 (+ 1381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:24.626222 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:24.626223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:24.626223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:24.626276 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:24.626442 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:24.626442 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:24.626443 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:24.626576 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:24.626719 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:24.626791 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 08:52:24.626854 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 08:52:24.628419 15691 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35994:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:52:24.629994 15691 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:52:24.636831 16477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:24.632763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40912 (local address 127.8.154.1:43125)
0809 08:52:24.632921 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:24.632949 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:24.632959 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:52:24.633016 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:24.633019 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:24.633036 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:24.633330 (+ 294us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:24.633407 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:24.634173 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:24.635721 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:24.636364 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:24.636464 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:24.636527 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:24.636692 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 08:52:24.636733 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":92,"threads_started":1}
I20260809 08:52:24.636862 16469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:24.632493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:43125 (local address 127.0.0.1:40912)
0809 08:52:24.632658 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:24.632702 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:24.632772 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:24.633157 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:24.633161 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:24.633164 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:24.633246 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:24.633249 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:24.634276 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:24.634277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:24.635618 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:24.635621 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:24.636468 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:24.636469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:24.636559 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:24.636574 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 08:52:24.636672 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 08:52:24.636737 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":30,"thread_start_us":107,"threads_started":1}
I20260809 08:52:24.636974 16473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:24.632665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:46131 (local address 127.0.0.1:45606)
0809 08:52:24.633099 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:24.633109 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:24.633154 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:24.633490 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:24.633492 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:24.633494 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:24.633532 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:24.633533 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:24.634715 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:24.634716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:24.635792 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:24.635794 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:24.636774 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:24.636775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:24.636831 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:24.636834 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:52:24.636882 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 08:52:24.636919 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":90,"threads_started":1}
I20260809 08:52:24.637038 16471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:24.632566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60658 (local address 127.8.154.2:39955)
0809 08:52:24.633003 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:24.633007 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:24.633016 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:52:24.633053 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:24.633055 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:24.633071 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:24.633145 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:24.633348 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:24.634482 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:24.635667 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:24.636424 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:24.636498 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:24.636552 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:24.636906 (+ 354us) server_negotiation.cc:299] Negotiation successful
0809 08:52:24.636949 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":72,"threads_started":1}
I20260809 08:52:24.637079 16470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:24.632440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:39955 (local address 127.0.0.1:60658)
0809 08:52:24.632658 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:24.632702 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:24.632767 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:24.633241 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:24.633242 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:24.633244 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:24.633280 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:24.633282 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:24.634596 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:24.634597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:24.635573 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:24.635576 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:24.636593 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:24.636594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:24.636668 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:24.636671 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:52:24.636720 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 08:52:24.636772 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":80,"threads_started":1}
I20260809 08:52:24.637281 16479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:24.632884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45606 (local address 127.8.154.3:46131)
0809 08:52:24.633319 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:24.633324 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:24.633335 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:52:24.633371 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:24.633373 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:24.633390 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:24.633476 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:24.633593 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:24.634614 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:24.635878 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:24.636612 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:24.636673 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:24.636740 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:24.637137 (+ 397us) server_negotiation.cc:299] Negotiation successful
0809 08:52:24.637186 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":128,"threads_started":1}
I20260809 08:52:24.637998 16351 tablet_service.cc:1467] Processing CreateTablet for tablet b6e7dcc05d2d4b06b8f9bf7aab9569af (DEFAULT_TABLE table=test-table [id=212852167a5248579e7205b67e1fe7b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:24.638046 16136 tablet_service.cc:1467] Processing CreateTablet for tablet b6e7dcc05d2d4b06b8f9bf7aab9569af (DEFAULT_TABLE table=test-table [id=212852167a5248579e7205b67e1fe7b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:24.638382 16351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6e7dcc05d2d4b06b8f9bf7aab9569af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:24.638382 16136 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6e7dcc05d2d4b06b8f9bf7aab9569af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:24.638275 15918 tablet_service.cc:1467] Processing CreateTablet for tablet b6e7dcc05d2d4b06b8f9bf7aab9569af (DEFAULT_TABLE table=test-table [id=212852167a5248579e7205b67e1fe7b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:24.638643 15918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6e7dcc05d2d4b06b8f9bf7aab9569af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:24.641266 16480 tablet_bootstrap.cc:492] T b6e7dcc05d2d4b06b8f9bf7aab9569af P bd832ae748a64b6683993285708076ea: Bootstrap starting.
I20260809 08:52:24.641907 16481 tablet_bootstrap.cc:492] T b6e7dcc05d2d4b06b8f9bf7aab9569af P ef0162763bdb43d5ab385ee232fd1482: Bootstrap starting.
I20260809 08:52:24.641964 16483 tablet_bootstrap.cc:492] T b6e7dcc05d2d4b06b8f9bf7aab9569af P 2a6b71984e1c41c295a30720403128a7: Bootstrap starting.
I20260809 08:52:24.642079 16480 tablet_bootstrap.cc:654] T b6e7dcc05d2d4b06b8f9bf7aab9569af P bd832ae748a64b6683993285708076ea: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:24.642553 16483 tablet_bootstrap.cc:654] T b6e7dcc05d2d4b06b8f9bf7aab9569af P 2a6b71984e1c41c295a30720403128a7: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:24.642704 16481 tablet_bootstrap.cc:654] T b6e7dcc05d2d4b06b8f9bf7aab9569af P ef0162763bdb43d5ab385ee232fd1482: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:24.642904 16480 log.cc:824] T b6e7dcc05d2d4b06b8f9bf7aab9569af P bd832ae748a64b6683993285708076ea: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:24.643211 16483 log.cc:824] T b6e7dcc05d2d4b06b8f9bf7aab9569af P 2a6b71984e1c41c295a30720403128a7: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:24.643321 16481 log.cc:824] T b6e7dcc05d2d4b06b8f9bf7aab9569af P ef0162763bdb43d5ab385ee232fd1482: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:24.643872 16480 tablet_bootstrap.cc:492] T b6e7dcc05d2d4b06b8f9bf7aab9569af P bd832ae748a64b6683993285708076ea: No bootstrap required, opened a new log
I20260809 08:52:24.643952 16480 ts_tablet_manager.cc:1397] T b6e7dcc05d2d4b06b8f9bf7aab9569af P bd832ae748a64b6683993285708076ea: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 08:52:24.644131 16481 tablet_bootstrap.cc:492] T b6e7dcc05d2d4b06b8f9bf7aab9569af P ef0162763bdb43d5ab385ee232fd1482: No bootstrap required, opened a new log
I20260809 08:52:24.644202 16481 ts_tablet_manager.cc:1397] T b6e7dcc05d2d4b06b8f9bf7aab9569af P ef0162763bdb43d5ab385ee232fd1482: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:52:24.644557 16483 tablet_bootstrap.cc:492] T b6e7dcc05d2d4b06b8f9bf7aab9569af P 2a6b71984e1c41c295a30720403128a7: No bootstrap required, opened a new log
I20260809 08:52:24.644645 16483 ts_tablet_manager.cc:1397] T b6e7dcc05d2d4b06b8f9bf7aab9569af P 2a6b71984e1c41c295a30720403128a7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 08:52:24.646063 16480 raft_consensus.cc:348] T b6e7dcc05d2d4b06b8f9bf7aab9569af P bd832ae748a64b6683993285708076ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6b71984e1c41c295a30720403128a7" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 39955 } } peers { permanent_uuid: "bd832ae748a64b6683993285708076ea" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43125 } } peers { permanent_uuid: "ef0162763bdb43d5ab385ee232fd1482" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 46131 } }
I20260809 08:52:24.646242 16480 raft_consensus.cc:374] T b6e7dcc05d2d4b06b8f9bf7aab9569af P bd832ae748a64b6683993285708076ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:24.646286 16480 raft_consensus.cc:729] T b6e7dcc05d2d4b06b8f9bf7aab9569af P bd832ae748a64b6683993285708076ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd832ae748a64b6683993285708076ea, State: Initialized, Role: FOLLOWER
I20260809 08:52:24.646279 16481 raft_consensus.cc:348] T b6e7dcc05d2d4b06b8f9bf7aab9569af P ef0162763bdb43d5ab385ee232fd1482 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6b71984e1c41c295a30720403128a7" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 39955 } } peers { permanent_uuid: "bd832ae748a64b6683993285708076ea" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43125 } } peers { permanent_uuid: "ef0162763bdb43d5ab385ee232fd1482" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 46131 } }
I20260809 08:52:24.646263 16483 raft_consensus.cc:348] T b6e7dcc05d2d4b06b8f9bf7aab9569af P 2a6b71984e1c41c295a30720403128a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6b71984e1c41c295a30720403128a7" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 39955 } } peers { permanent_uuid: "bd832ae748a64b6683993285708076ea" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43125 } } peers { permanent_uuid: "ef0162763bdb43d5ab385ee232fd1482" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 46131 } }
I20260809 08:52:24.646415 16481 raft_consensus.cc:374] T b6e7dcc05d2d4b06b8f9bf7aab9569af P ef0162763bdb43d5ab385ee232fd1482 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:24.646400 16480 consensus_queue.cc:260] T b6e7dcc05d2d4b06b8f9bf7aab9569af P bd832ae748a64b6683993285708076ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6b71984e1c41c295a30720403128a7" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 39955 } } peers { permanent_uuid: "bd832ae748a64b6683993285708076ea" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43125 } } peers { permanent_uuid: "ef0162763bdb43d5ab385ee232fd1482" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 46131 } }
I20260809 08:52:24.646447 16481 raft_consensus.cc:729] T b6e7dcc05d2d4b06b8f9bf7aab9569af P ef0162763bdb43d5ab385ee232fd1482 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef0162763bdb43d5ab385ee232fd1482, State: Initialized, Role: FOLLOWER
I20260809 08:52:24.646450 16483 raft_consensus.cc:374] T b6e7dcc05d2d4b06b8f9bf7aab9569af P 2a6b71984e1c41c295a30720403128a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:24.646489 16483 raft_consensus.cc:729] T b6e7dcc05d2d4b06b8f9bf7aab9569af P 2a6b71984e1c41c295a30720403128a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a6b71984e1c41c295a30720403128a7, State: Initialized, Role: FOLLOWER
I20260809 08:52:24.646576 16481 consensus_queue.cc:260] T b6e7dcc05d2d4b06b8f9bf7aab9569af P ef0162763bdb43d5ab385ee232fd1482 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6b71984e1c41c295a30720403128a7" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 39955 } } peers { permanent_uuid: "bd832ae748a64b6683993285708076ea" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43125 } } peers { permanent_uuid: "ef0162763bdb43d5ab385ee232fd1482" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 46131 } }
I20260809 08:52:24.646595 16483 consensus_queue.cc:260] T b6e7dcc05d2d4b06b8f9bf7aab9569af P 2a6b71984e1c41c295a30720403128a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6b71984e1c41c295a30720403128a7" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 39955 } } peers { permanent_uuid: "bd832ae748a64b6683993285708076ea" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43125 } } peers { permanent_uuid: "ef0162763bdb43d5ab385ee232fd1482" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 46131 } }
I20260809 08:52:24.646682 16480 ts_tablet_manager.cc:1428] T b6e7dcc05d2d4b06b8f9bf7aab9569af P bd832ae748a64b6683993285708076ea: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:52:24.646842 16028 heartbeater.cc:503] Master 127.8.154.62:32863 was elected leader, sending a full tablet report...
I20260809 08:52:24.646874 16483 ts_tablet_manager.cc:1428] T b6e7dcc05d2d4b06b8f9bf7aab9569af P 2a6b71984e1c41c295a30720403128a7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:52:24.646881 16481 ts_tablet_manager.cc:1428] T b6e7dcc05d2d4b06b8f9bf7aab9569af P ef0162763bdb43d5ab385ee232fd1482: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:52:24.647012 16449 heartbeater.cc:503] Master 127.8.154.62:32863 was elected leader, sending a full tablet report...
I20260809 08:52:24.647011 16245 heartbeater.cc:503] Master 127.8.154.62:32863 was elected leader, sending a full tablet report...
I20260809 08:52:24.672317 16490 raft_consensus.cc:482] T b6e7dcc05d2d4b06b8f9bf7aab9569af P bd832ae748a64b6683993285708076ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:52:24.672432 16490 raft_consensus.cc:504] T b6e7dcc05d2d4b06b8f9bf7aab9569af P bd832ae748a64b6683993285708076ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6b71984e1c41c295a30720403128a7" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 39955 } } peers { permanent_uuid: "bd832ae748a64b6683993285708076ea" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43125 } } peers { permanent_uuid: "ef0162763bdb43d5ab385ee232fd1482" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 46131 } }
I20260809 08:52:24.672950 16490 leader_election.cc:288] T b6e7dcc05d2d4b06b8f9bf7aab9569af P bd832ae748a64b6683993285708076ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a6b71984e1c41c295a30720403128a7 (127.8.154.2:39955), ef0162763bdb43d5ab385ee232fd1482 (127.8.154.3:46131)
I20260809 08:52:24.677125 16471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:24.673256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:35307 (local address 127.8.154.2:39955)
0809 08:52:24.673309 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:24.673312 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:24.673324 (+ 12us) server_negotiation.cc:407] Connection header received
0809 08:52:24.673367 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:24.673368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:24.673380 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:24.673458 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:24.673719 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:24.674486 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:24.676112 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:24.676728 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:24.676795 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:24.676812 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:24.677008 (+ 196us) server_negotiation.cc:299] Negotiation successful
0809 08:52:24.677057 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 08:52:24.677127 16479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:24.673354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:58601 (local address 127.8.154.3:46131)
0809 08:52:24.673413 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:24.673415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:24.673546 (+ 131us) server_negotiation.cc:407] Connection header received
0809 08:52:24.673609 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:24.673610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:24.673619 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:24.673689 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:24.673833 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:24.674513 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:24.676123 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:24.676708 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:24.676795 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:24.676807 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:24.677012 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 08:52:24.677059 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 08:52:24.677140 16030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:24.673149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:39955 (local address 127.8.154.1:35307)
0809 08:52:24.673223 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:24.673238 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:24.673299 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:24.673540 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:24.673541 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:24.673543 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:24.673624 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:24.673626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:24.674612 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:24.674614 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:24.676000 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:24.676004 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:24.676813 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:24.676814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:24.676902 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:24.676904 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 08:52:24.676963 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 08:52:24.677023 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":123}
I20260809 08:52:24.677148 16495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:24.673277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:46131 (local address 127.8.154.1:58601)
0809 08:52:24.673473 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:24.673484 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:24.673544 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:24.673698 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:24.673699 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:24.673701 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:24.673769 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:24.673770 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:24.674612 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:24.674614 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:24.676000 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:24.676004 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:24.676813 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:24.676814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:24.676896 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:24.676900 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:52:24.676963 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 08:52:24.677023 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":56,"thread_start_us":126,"threads_started":1}
I20260809 08:52:24.677516 16169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6e7dcc05d2d4b06b8f9bf7aab9569af" candidate_uuid: "bd832ae748a64b6683993285708076ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a6b71984e1c41c295a30720403128a7" is_pre_election: true
I20260809 08:52:24.677677 16169 raft_consensus.cc:2454] T b6e7dcc05d2d4b06b8f9bf7aab9569af P 2a6b71984e1c41c295a30720403128a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd832ae748a64b6683993285708076ea in term 0.
I20260809 08:52:24.677732 16386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6e7dcc05d2d4b06b8f9bf7aab9569af" candidate_uuid: "bd832ae748a64b6683993285708076ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef0162763bdb43d5ab385ee232fd1482" is_pre_election: true
I20260809 08:52:24.677870 16386 raft_consensus.cc:2454] T b6e7dcc05d2d4b06b8f9bf7aab9569af P ef0162763bdb43d5ab385ee232fd1482 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd832ae748a64b6683993285708076ea in term 0.
I20260809 08:52:24.677965 15826 leader_election.cc:302] T b6e7dcc05d2d4b06b8f9bf7aab9569af P bd832ae748a64b6683993285708076ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a6b71984e1c41c295a30720403128a7, bd832ae748a64b6683993285708076ea; no voters:
I20260809 08:52:24.678119 16490 raft_consensus.cc:2781] T b6e7dcc05d2d4b06b8f9bf7aab9569af P bd832ae748a64b6683993285708076ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:52:24.678179 16490 raft_consensus.cc:482] T b6e7dcc05d2d4b06b8f9bf7aab9569af P bd832ae748a64b6683993285708076ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:52:24.678206 16490 raft_consensus.cc:3037] T b6e7dcc05d2d4b06b8f9bf7aab9569af P bd832ae748a64b6683993285708076ea [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:24.678890 16490 raft_consensus.cc:504] T b6e7dcc05d2d4b06b8f9bf7aab9569af P bd832ae748a64b6683993285708076ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6b71984e1c41c295a30720403128a7" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 39955 } } peers { permanent_uuid: "bd832ae748a64b6683993285708076ea" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43125 } } peers { permanent_uuid: "ef0162763bdb43d5ab385ee232fd1482" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 46131 } }
I20260809 08:52:24.679031 16490 leader_election.cc:288] T b6e7dcc05d2d4b06b8f9bf7aab9569af P bd832ae748a64b6683993285708076ea [CANDIDATE]: Term 1 election: Requested vote from peers 2a6b71984e1c41c295a30720403128a7 (127.8.154.2:39955), ef0162763bdb43d5ab385ee232fd1482 (127.8.154.3:46131)
I20260809 08:52:24.679229 16386 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6e7dcc05d2d4b06b8f9bf7aab9569af" candidate_uuid: "bd832ae748a64b6683993285708076ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef0162763bdb43d5ab385ee232fd1482"
I20260809 08:52:24.679309 16386 raft_consensus.cc:3037] T b6e7dcc05d2d4b06b8f9bf7aab9569af P ef0162763bdb43d5ab385ee232fd1482 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:24.679273 16169 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6e7dcc05d2d4b06b8f9bf7aab9569af" candidate_uuid: "bd832ae748a64b6683993285708076ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a6b71984e1c41c295a30720403128a7"
I20260809 08:52:24.679364 16169 raft_consensus.cc:3037] T b6e7dcc05d2d4b06b8f9bf7aab9569af P 2a6b71984e1c41c295a30720403128a7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:24.680233 16386 raft_consensus.cc:2454] T b6e7dcc05d2d4b06b8f9bf7aab9569af P ef0162763bdb43d5ab385ee232fd1482 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd832ae748a64b6683993285708076ea in term 1.
I20260809 08:52:24.680405 15826 leader_election.cc:302] T b6e7dcc05d2d4b06b8f9bf7aab9569af P bd832ae748a64b6683993285708076ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bd832ae748a64b6683993285708076ea, ef0162763bdb43d5ab385ee232fd1482; no voters:
I20260809 08:52:24.680447 16169 raft_consensus.cc:2454] T b6e7dcc05d2d4b06b8f9bf7aab9569af P 2a6b71984e1c41c295a30720403128a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd832ae748a64b6683993285708076ea in term 1.
I20260809 08:52:24.680492 16490 raft_consensus.cc:2781] T b6e7dcc05d2d4b06b8f9bf7aab9569af P bd832ae748a64b6683993285708076ea [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:52:24.680658 16490 raft_consensus.cc:686] T b6e7dcc05d2d4b06b8f9bf7aab9569af P bd832ae748a64b6683993285708076ea [term 1 LEADER]: Becoming Leader. State: Replica: bd832ae748a64b6683993285708076ea, State: Running, Role: LEADER
I20260809 08:52:24.680776 16490 consensus_queue.cc:237] T b6e7dcc05d2d4b06b8f9bf7aab9569af P bd832ae748a64b6683993285708076ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6b71984e1c41c295a30720403128a7" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 39955 } } peers { permanent_uuid: "bd832ae748a64b6683993285708076ea" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43125 } } peers { permanent_uuid: "ef0162763bdb43d5ab385ee232fd1482" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 46131 } }
I20260809 08:52:24.681625 15690 catalog_manager.cc:5495] T b6e7dcc05d2d4b06b8f9bf7aab9569af P bd832ae748a64b6683993285708076ea reported cstate change: term changed from 0 to 1, leader changed from <none> to bd832ae748a64b6683993285708076ea (127.8.154.1). New cstate: current_term: 1 leader_uuid: "bd832ae748a64b6683993285708076ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6b71984e1c41c295a30720403128a7" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 39955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd832ae748a64b6683993285708076ea" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43125 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef0162763bdb43d5ab385ee232fd1482" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 46131 } health_report { overall_health: UNKNOWN } } }
I20260809 08:52:24.696898 16477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:24.694350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40924 (local address 127.8.154.1:43125)
0809 08:52:24.694405 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:24.694407 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:24.694418 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:52:24.694477 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:24.694478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:24.694498 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:24.694578 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:52:24.694745 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:24.695219 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:24.696006 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:24.696261 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:24.696342 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:24.696529 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:52:24.696635 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:52:24.696725 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:24.696766 (+ 41us) server_negotiation.cc:299] Negotiation successful
0809 08:52:24.696789 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 08:52:24.698900 16169 raft_consensus.cc:1261] T b6e7dcc05d2d4b06b8f9bf7aab9569af P 2a6b71984e1c41c295a30720403128a7 [term 1 FOLLOWER]: Refusing update from remote peer bd832ae748a64b6683993285708076ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:52:24.698900 16386 raft_consensus.cc:1261] T b6e7dcc05d2d4b06b8f9bf7aab9569af P ef0162763bdb43d5ab385ee232fd1482 [term 1 FOLLOWER]: Refusing update from remote peer bd832ae748a64b6683993285708076ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:52:24.699291 16490 consensus_queue.cc:1035] T b6e7dcc05d2d4b06b8f9bf7aab9569af P bd832ae748a64b6683993285708076ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef0162763bdb43d5ab385ee232fd1482" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 46131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:52:24.699456 16496 consensus_queue.cc:1035] T b6e7dcc05d2d4b06b8f9bf7aab9569af P bd832ae748a64b6683993285708076ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a6b71984e1c41c295a30720403128a7" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 39955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 08:52:24.701673 16246 tablet.cc:2369] T b6e7dcc05d2d4b06b8f9bf7aab9569af P 2a6b71984e1c41c295a30720403128a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:52:24.701901 16501 mvcc.cc:205] Tried to move back new op lower bound from 7316543671083532288 to 7316543671012986880. Current Snapshot: MvccSnapshot[applied={T|T < 7316543671083532288}]
I20260809 08:52:24.702003 16502 mvcc.cc:205] Tried to move back new op lower bound from 7316543671083532288 to 7316543671012986880. Current Snapshot: MvccSnapshot[applied={T|T < 7316543671083532288}]
I20260809 08:52:24.702630 16503 mvcc.cc:205] Tried to move back new op lower bound from 7316543671083532288 to 7316543671012986880. Current Snapshot: MvccSnapshot[applied={T|T < 7316543671083532288}]
I20260809 08:52:24.704897 15691 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35994:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:52:24.704988 15691 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35994:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:52:24.705688 15691 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 440ec2dd16514576b5dd07fa590ba5cc: Sending DeleteTablet for 3 replicas of tablet b6e7dcc05d2d4b06b8f9bf7aab9569af
I20260809 08:52:24.706013 15918 tablet_service.cc:1514] Processing DeleteTablet for tablet b6e7dcc05d2d4b06b8f9bf7aab9569af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:24 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:40912
I20260809 08:52:24.706065 16351 tablet_service.cc:1514] Processing DeleteTablet for tablet b6e7dcc05d2d4b06b8f9bf7aab9569af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:24 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:45606
I20260809 08:52:24.706104 16136 tablet_service.cc:1514] Processing DeleteTablet for tablet b6e7dcc05d2d4b06b8f9bf7aab9569af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:24 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:60658
I20260809 08:52:24.706328 16517 tablet_replica.cc:330] T b6e7dcc05d2d4b06b8f9bf7aab9569af P bd832ae748a64b6683993285708076ea: stopping tablet replica
I20260809 08:52:24.706435 16517 raft_consensus.cc:2229] T b6e7dcc05d2d4b06b8f9bf7aab9569af P bd832ae748a64b6683993285708076ea [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:52:24.706548 16517 raft_consensus.cc:2258] T b6e7dcc05d2d4b06b8f9bf7aab9569af P bd832ae748a64b6683993285708076ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:52:24.706632 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 15797
I20260809 08:52:24.706702 16518 tablet_replica.cc:330] T b6e7dcc05d2d4b06b8f9bf7aab9569af P ef0162763bdb43d5ab385ee232fd1482: stopping tablet replica
I20260809 08:52:24.706799 16518 raft_consensus.cc:2229] T b6e7dcc05d2d4b06b8f9bf7aab9569af P ef0162763bdb43d5ab385ee232fd1482 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:52:24.706889 16518 raft_consensus.cc:2258] T b6e7dcc05d2d4b06b8f9bf7aab9569af P ef0162763bdb43d5ab385ee232fd1482 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:52:24.707495 16518 ts_tablet_manager.cc:1905] T b6e7dcc05d2d4b06b8f9bf7aab9569af P ef0162763bdb43d5ab385ee232fd1482: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:52:24.709717 16518 ts_tablet_manager.cc:1918] T b6e7dcc05d2d4b06b8f9bf7aab9569af P ef0162763bdb43d5ab385ee232fd1482: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:52:24.709796 16518 log.cc:1196] T b6e7dcc05d2d4b06b8f9bf7aab9569af P ef0162763bdb43d5ab385ee232fd1482: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-2/wal/wals/b6e7dcc05d2d4b06b8f9bf7aab9569af
I20260809 08:52:24.710052 16518 ts_tablet_manager.cc:1939] T b6e7dcc05d2d4b06b8f9bf7aab9569af P ef0162763bdb43d5ab385ee232fd1482: Deleting consensus metadata
I20260809 08:52:24.710465 15669 catalog_manager.cc:4841] TS ef0162763bdb43d5ab385ee232fd1482 (127.8.154.3:46131): tablet b6e7dcc05d2d4b06b8f9bf7aab9569af (table test-table [id=212852167a5248579e7205b67e1fe7b4]) successfully deleted
I20260809 08:52:24.713492 16520 tablet_replica.cc:330] T b6e7dcc05d2d4b06b8f9bf7aab9569af P 2a6b71984e1c41c295a30720403128a7: stopping tablet replica
I20260809 08:52:24.713621 16520 raft_consensus.cc:2229] T b6e7dcc05d2d4b06b8f9bf7aab9569af P 2a6b71984e1c41c295a30720403128a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:52:24.713730 16520 raft_consensus.cc:2258] T b6e7dcc05d2d4b06b8f9bf7aab9569af P 2a6b71984e1c41c295a30720403128a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:52:24.715346 16520 ts_tablet_manager.cc:1905] T b6e7dcc05d2d4b06b8f9bf7aab9569af P 2a6b71984e1c41c295a30720403128a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:52:24.716598 16520 ts_tablet_manager.cc:1918] T b6e7dcc05d2d4b06b8f9bf7aab9569af P 2a6b71984e1c41c295a30720403128a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:52:24.716666 16520 log.cc:1196] T b6e7dcc05d2d4b06b8f9bf7aab9569af P 2a6b71984e1c41c295a30720403128a7: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265519942966-8808-0/minicluster-data/ts-1/wal/wals/b6e7dcc05d2d4b06b8f9bf7aab9569af
I20260809 08:52:24.716928 16520 ts_tablet_manager.cc:1939] T b6e7dcc05d2d4b06b8f9bf7aab9569af P 2a6b71984e1c41c295a30720403128a7: Deleting consensus metadata
I20260809 08:52:24.717340 15669 catalog_manager.cc:4841] TS 2a6b71984e1c41c295a30720403128a7 (127.8.154.2:39955): tablet b6e7dcc05d2d4b06b8f9bf7aab9569af (table test-table [id=212852167a5248579e7205b67e1fe7b4]) successfully deleted
W20260809 08:52:24.719844 15667 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.154.1:43125 (error 108)
I20260809 08:52:24.720610 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 16035
I20260809 08:52:24.720649 16473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:24.720437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:43125 (local address 127.0.0.1:40938)
0809 08:52:24.720541 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:24.720605 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.1:43125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260809 08:52:24.720790 15667 catalog_manager.cc:4584] TS bd832ae748a64b6683993285708076ea (127.8.154.1:43125): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b6e7dcc05d2d4b06b8f9bf7aab9569af: Network error: Client connection negotiation failed: client connection to 127.8.154.1:43125: connect: Connection refused (error 111)
I20260809 08:52:24.732151 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 16250
I20260809 08:52:24.746413 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 15646
2026-08-09T08:52:24Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2191 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T08:52:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:52:24Z Disabled control of system clock
I20260809 08:52:24.780853 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:36167
--webserver_interface=127.8.154.62
--webserver_port=0
--builtin_ntp_servers=127.8.154.20:46413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:36167
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 08:52:24.863739 16527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:24.863941 16527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:24.863963 16527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:24.865139 16527 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:52:24.865186 16527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:24.865199 16527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:24.865212 16527 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:52:24.865224 16527 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:52:24.866524 16527 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:46413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:36167
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:36167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:24.866727 16527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:24.866976 16527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:24.872331 16534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:24.872377 16537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:24.872535 16527 server_base.cc:1034] running on GCE node
W20260809 08:52:24.872646 16533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:24.872952 16527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:24.873337 16527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:24.874552 16527 hybrid_clock.cc:648] HybridClock initialized: now 1786265544874527 us; error 34 us; skew 500 ppm
I20260809 08:52:24.875907 16527 webserver.cc:460] Webserver started at http://127.8.154.62:46823/ using document root <none> and password file <none>
I20260809 08:52:24.876240 16527 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:24.876314 16527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:24.876477 16527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:24.877658 16527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "f14b6a42e5b8485ba2e2bd88fadbf4c3"
format_stamp: "Formatted at 2026-08-09 08:52:24 on dist-test-slave-t3q3"
server_key: "e3e6ff365e54a2f04ecd9bfc5f457976"
server_key_iv: "af77ad31241e995b644c20f3cb2ad9e8"
server_key_version: "encryptionkey@0"
I20260809 08:52:24.878063 16527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "f14b6a42e5b8485ba2e2bd88fadbf4c3"
format_stamp: "Formatted at 2026-08-09 08:52:24 on dist-test-slave-t3q3"
server_key: "e3e6ff365e54a2f04ecd9bfc5f457976"
server_key_iv: "af77ad31241e995b644c20f3cb2ad9e8"
server_key_version: "encryptionkey@0"
I20260809 08:52:24.879428 16527 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 08:52:24.880271 16545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:24.880435 16527 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:24.880544 16527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "f14b6a42e5b8485ba2e2bd88fadbf4c3"
format_stamp: "Formatted at 2026-08-09 08:52:24 on dist-test-slave-t3q3"
server_key: "e3e6ff365e54a2f04ecd9bfc5f457976"
server_key_iv: "af77ad31241e995b644c20f3cb2ad9e8"
server_key_version: "encryptionkey@0"
I20260809 08:52:24.880625 16527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:24.884016 16527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:24.884343 16527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:24.884492 16527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:24.888586 16527 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:36167
I20260809 08:52:24.888635 16631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:36167 every 8 connection(s)
I20260809 08:52:24.888995 16527 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:52:24.889734 16632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:24.892719 16632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f14b6a42e5b8485ba2e2bd88fadbf4c3: Bootstrap starting.
I20260809 08:52:24.893342 16632 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f14b6a42e5b8485ba2e2bd88fadbf4c3: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:24.893648 16632 log.cc:824] T 00000000000000000000000000000000 P f14b6a42e5b8485ba2e2bd88fadbf4c3: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:24.894349 16632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f14b6a42e5b8485ba2e2bd88fadbf4c3: No bootstrap required, opened a new log
I20260809 08:52:24.895592 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 16527
I20260809 08:52:24.895716 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:52:24.895912 8808 external_mini_cluster.cc:1428] Setting key c9ccd51c747e88da64e7b1d6756f535c
I20260809 08:52:24.896100 16632 raft_consensus.cc:348] T 00000000000000000000000000000000 P f14b6a42e5b8485ba2e2bd88fadbf4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f14b6a42e5b8485ba2e2bd88fadbf4c3" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 36167 } }
I20260809 08:52:24.896287 16632 raft_consensus.cc:374] T 00000000000000000000000000000000 P f14b6a42e5b8485ba2e2bd88fadbf4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:24.896349 16632 raft_consensus.cc:729] T 00000000000000000000000000000000 P f14b6a42e5b8485ba2e2bd88fadbf4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f14b6a42e5b8485ba2e2bd88fadbf4c3, State: Initialized, Role: FOLLOWER
I20260809 08:52:24.896482 16632 consensus_queue.cc:260] T 00000000000000000000000000000000 P f14b6a42e5b8485ba2e2bd88fadbf4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f14b6a42e5b8485ba2e2bd88fadbf4c3" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 36167 } }
I20260809 08:52:24.896587 16632 raft_consensus.cc:388] T 00000000000000000000000000000000 P f14b6a42e5b8485ba2e2bd88fadbf4c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:52:24.896636 16632 raft_consensus.cc:482] T 00000000000000000000000000000000 P f14b6a42e5b8485ba2e2bd88fadbf4c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:52:24.896689 16632 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f14b6a42e5b8485ba2e2bd88fadbf4c3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:24.897315 16632 raft_consensus.cc:504] T 00000000000000000000000000000000 P f14b6a42e5b8485ba2e2bd88fadbf4c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f14b6a42e5b8485ba2e2bd88fadbf4c3" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 36167 } }
I20260809 08:52:24.897437 16632 leader_election.cc:302] T 00000000000000000000000000000000 P f14b6a42e5b8485ba2e2bd88fadbf4c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f14b6a42e5b8485ba2e2bd88fadbf4c3; no voters:
I20260809 08:52:24.897611 16632 leader_election.cc:288] T 00000000000000000000000000000000 P f14b6a42e5b8485ba2e2bd88fadbf4c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:52:24.897804 16641 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f14b6a42e5b8485ba2e2bd88fadbf4c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:52:24.897918 16632 sys_catalog.cc:564] T 00000000000000000000000000000000 P f14b6a42e5b8485ba2e2bd88fadbf4c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:52:24.897960 16641 raft_consensus.cc:686] T 00000000000000000000000000000000 P f14b6a42e5b8485ba2e2bd88fadbf4c3 [term 1 LEADER]: Becoming Leader. State: Replica: f14b6a42e5b8485ba2e2bd88fadbf4c3, State: Running, Role: LEADER
I20260809 08:52:24.898041 16641 consensus_queue.cc:237] T 00000000000000000000000000000000 P f14b6a42e5b8485ba2e2bd88fadbf4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f14b6a42e5b8485ba2e2bd88fadbf4c3" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 36167 } }
I20260809 08:52:24.898413 16644 sys_catalog.cc:455] T 00000000000000000000000000000000 P f14b6a42e5b8485ba2e2bd88fadbf4c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f14b6a42e5b8485ba2e2bd88fadbf4c3. Latest consensus state: current_term: 1 leader_uuid: "f14b6a42e5b8485ba2e2bd88fadbf4c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f14b6a42e5b8485ba2e2bd88fadbf4c3" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 36167 } } }
I20260809 08:52:24.898497 16644 sys_catalog.cc:458] T 00000000000000000000000000000000 P f14b6a42e5b8485ba2e2bd88fadbf4c3 [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:24.898636 16643 sys_catalog.cc:455] T 00000000000000000000000000000000 P f14b6a42e5b8485ba2e2bd88fadbf4c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f14b6a42e5b8485ba2e2bd88fadbf4c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f14b6a42e5b8485ba2e2bd88fadbf4c3" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 36167 } } }
I20260809 08:52:24.898705 16643 sys_catalog.cc:458] T 00000000000000000000000000000000 P f14b6a42e5b8485ba2e2bd88fadbf4c3 [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:24.898977 16654 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:52:24.899021 16638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:24.896566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41858 (local address 127.8.154.62:36167)
0809 08:52:24.896810 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:24.896819 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:24.896859 (+ 40us) server_negotiation.cc:407] Connection header received
0809 08:52:24.896963 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:24.896968 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:24.897002 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:24.897125 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:24.898002 (+ 877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:24.898005 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:24.898006 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:52:24.898012 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:24.898069 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:52:24.898089 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:24.898138 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:24.898371 (+ 233us) server_negotiation.cc:299] Negotiation successful
0809 08:52:24.898440 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":99,"threads_started":1}
I20260809 08:52:24.899518 16654 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:52:24.904314 16654 catalog_manager.cc:1347] Generated new cluster ID: ae9dbbf6380d4c20a62997c7f34e6291
I20260809 08:52:24.904384 16654 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:52:24.910305 16654 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:52:24.910471 16654 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:52:24.917624 16654 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f14b6a42e5b8485ba2e2bd88fadbf4c3: Generated new TSK 0
I20260809 08:52:24.917845 16654 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:52:24.922363 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.1:0
--local_ip_for_outbound_sockets=127.8.154.1
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:36167
--builtin_ntp_servers=127.8.154.20:46413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 08:52:25.019989 16666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:25.020247 16666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:25.020270 16666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:25.021577 16666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:25.021622 16666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:25.021648 16666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.1
I20260809 08:52:25.023032 16666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:46413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:36167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.1
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:25.023270 16666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:25.023583 16666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:25.029672 16679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:25.029712 16681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:25.029698 16677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:25.030380 16666 server_base.cc:1034] running on GCE node
I20260809 08:52:25.030637 16666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:25.030923 16666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:25.032131 16666 hybrid_clock.cc:648] HybridClock initialized: now 1786265545032118 us; error 28 us; skew 500 ppm
I20260809 08:52:25.033704 16666 webserver.cc:460] Webserver started at http://127.8.154.1:38255/ using document root <none> and password file <none>
I20260809 08:52:25.033978 16666 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:25.034058 16666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:25.034202 16666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:25.035677 16666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-0/data/instance:
uuid: "54f26aa30dcf4c1dbab883cee04efc70"
format_stamp: "Formatted at 2026-08-09 08:52:25 on dist-test-slave-t3q3"
server_key: "68b4b57c553cdec63552ae65cfd4a853"
server_key_iv: "7a030cb3a320e8bea426e356cc92a567"
server_key_version: "encryptionkey@0"
I20260809 08:52:25.036151 16666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance:
uuid: "54f26aa30dcf4c1dbab883cee04efc70"
format_stamp: "Formatted at 2026-08-09 08:52:25 on dist-test-slave-t3q3"
server_key: "68b4b57c553cdec63552ae65cfd4a853"
server_key_iv: "7a030cb3a320e8bea426e356cc92a567"
server_key_version: "encryptionkey@0"
I20260809 08:52:25.037763 16666 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:52:25.038663 16688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:25.038875 16666 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:25.038969 16666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-0/wal
uuid: "54f26aa30dcf4c1dbab883cee04efc70"
format_stamp: "Formatted at 2026-08-09 08:52:25 on dist-test-slave-t3q3"
server_key: "68b4b57c553cdec63552ae65cfd4a853"
server_key_iv: "7a030cb3a320e8bea426e356cc92a567"
server_key_version: "encryptionkey@0"
I20260809 08:52:25.039047 16666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:25.043704 16666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:25.044098 16666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:25.044238 16666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:25.044454 16666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:25.044790 16666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:25.044852 16666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:25.044880 16666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:25.044898 16666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:25.052762 16666 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.1:39951
I20260809 08:52:25.052798 16862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.1:39951 every 8 connection(s)
I20260809 08:52:25.053216 16666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
I20260809 08:52:25.056700 16638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:25.054082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:54897 (local address 127.8.154.62:36167)
0809 08:52:25.054147 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:25.054150 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:25.054448 (+ 298us) server_negotiation.cc:407] Connection header received
0809 08:52:25.054989 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:25.054991 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:25.054999 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:25.055040 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:25.056431 (+ 1391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:25.056432 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:25.056433 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:52:25.056438 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:25.056456 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:52:25.056460 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:25.056516 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:25.056607 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 08:52:25.056625 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 08:52:25.056911 16865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:25.054071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:36167 (local address 127.8.154.1:54897)
0809 08:52:25.054320 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:25.054355 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:25.054871 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:25.055134 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:25.055137 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:25.055145 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:25.056270 (+ 1125us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:52:25.056270 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:25.056302 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:52:25.056331 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:25.056511 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:25.056516 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:25.056521 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:52:25.056618 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 08:52:25.056683 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":142,"threads_started":1}
I20260809 08:52:25.057394 16863 heartbeater.cc:347] Connected to a master server at 127.8.154.62:36167
I20260809 08:52:25.057488 16863 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:25.057650 16863 heartbeater.cc:511] Master 127.8.154.62:36167 requested a full tablet report, sending...
I20260809 08:52:25.058166 16569 ts_manager.cc:194] Registered new tserver with Master: 54f26aa30dcf4c1dbab883cee04efc70 (127.8.154.1:39951)
I20260809 08:52:25.058893 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 16666
I20260809 08:52:25.059000 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance
I20260809 08:52:25.059201 8808 external_mini_cluster.cc:1428] Setting key 429e9f567f16f4ec1f78844fe5fe8279
I20260809 08:52:25.060215 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.2:0
--local_ip_for_outbound_sockets=127.8.154.2
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:36167
--builtin_ntp_servers=127.8.154.20:46413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 08:52:25.144508 16868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:25.144723 16868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:25.144747 16868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:25.145922 16868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:25.145965 16868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:25.145991 16868 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.2
I20260809 08:52:25.147375 16868 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:46413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:36167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.2
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:25.147650 16868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:25.147959 16868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 08:52:25.153398 16868 server_base.cc:1034] running on GCE node
W20260809 08:52:25.153374 16880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:25.153357 16882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:25.153365 16879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:25.153735 16868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:25.153965 16868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:25.155117 16868 hybrid_clock.cc:648] HybridClock initialized: now 1786265545155100 us; error 30 us; skew 500 ppm
I20260809 08:52:25.156467 16868 webserver.cc:460] Webserver started at http://127.8.154.2:35121/ using document root <none> and password file <none>
I20260809 08:52:25.156723 16868 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:25.156774 16868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:25.156952 16868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:25.158066 16868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-1/data/instance:
uuid: "8afe19e4c8a94686869407da039e36db"
format_stamp: "Formatted at 2026-08-09 08:52:25 on dist-test-slave-t3q3"
server_key: "682b4be49cf3a4d2d52d8ea8b1568c84"
server_key_iv: "95873320d41df6b128afb95f8aaef3d5"
server_key_version: "encryptionkey@0"
I20260809 08:52:25.158442 16868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance:
uuid: "8afe19e4c8a94686869407da039e36db"
format_stamp: "Formatted at 2026-08-09 08:52:25 on dist-test-slave-t3q3"
server_key: "682b4be49cf3a4d2d52d8ea8b1568c84"
server_key_iv: "95873320d41df6b128afb95f8aaef3d5"
server_key_version: "encryptionkey@0"
I20260809 08:52:25.159965 16868 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:52:25.160866 16892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:25.161028 16868 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:52:25.161141 16868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-1/wal
uuid: "8afe19e4c8a94686869407da039e36db"
format_stamp: "Formatted at 2026-08-09 08:52:25 on dist-test-slave-t3q3"
server_key: "682b4be49cf3a4d2d52d8ea8b1568c84"
server_key_iv: "95873320d41df6b128afb95f8aaef3d5"
server_key_version: "encryptionkey@0"
I20260809 08:52:25.161216 16868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:25.164651 16868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:25.164928 16868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:25.165076 16868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:25.165299 16868 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:25.165638 16868 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:25.165674 16868 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:25.165695 16868 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:25.165709 16868 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:25.171862 16868 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.2:40331
I20260809 08:52:25.171901 17085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.2:40331 every 8 connection(s)
I20260809 08:52:25.172320 16868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
I20260809 08:52:25.175345 16638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:25.173141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:37593 (local address 127.8.154.62:36167)
0809 08:52:25.173227 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:25.173231 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:25.173477 (+ 246us) server_negotiation.cc:407] Connection header received
0809 08:52:25.173942 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:25.173943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:25.173948 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:25.174003 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:25.175114 (+ 1111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:25.175115 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:25.175115 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:52:25.175119 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:25.175134 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:52:25.175139 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:25.175189 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:25.175276 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 08:52:25.175290 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 08:52:25.175527 17090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:25.173125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:36167 (local address 127.8.154.2:37593)
0809 08:52:25.173353 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:25.173389 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:25.173856 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:25.174059 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:25.174062 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:25.174070 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:25.174965 (+ 895us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:52:25.174966 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:25.174997 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:52:25.175028 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:25.175202 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:25.175207 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:25.175210 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:52:25.175272 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 08:52:25.175323 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":118,"threads_started":1}
I20260809 08:52:25.175740 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 16868
I20260809 08:52:25.175848 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance
I20260809 08:52:25.176041 8808 external_mini_cluster.cc:1428] Setting key 420161ceb6d98ef8ff07a4829b7ca6ae
I20260809 08:52:25.176025 17087 heartbeater.cc:347] Connected to a master server at 127.8.154.62:36167
I20260809 08:52:25.176186 17087 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:25.176369 17087 heartbeater.cc:511] Master 127.8.154.62:36167 requested a full tablet report, sending...
I20260809 08:52:25.176733 16569 ts_manager.cc:194] Registered new tserver with Master: 8afe19e4c8a94686869407da039e36db (127.8.154.2:40331)
I20260809 08:52:25.177264 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.3:0
--local_ip_for_outbound_sockets=127.8.154.3
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:36167
--builtin_ntp_servers=127.8.154.20:46413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 08:52:25.262178 17091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:25.262391 17091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:25.262416 17091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:25.263581 17091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:25.263635 17091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:25.263734 17091 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.3
I20260809 08:52:25.265268 17091 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:46413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:36167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.3
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:25.265532 17091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:25.265841 17091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:25.270955 17096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:25.271046 17097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:25.271063 17100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:25.271330 17091 server_base.cc:1034] running on GCE node
I20260809 08:52:25.271559 17091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:25.271809 17091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:25.272964 17091 hybrid_clock.cc:648] HybridClock initialized: now 1786265545272932 us; error 48 us; skew 500 ppm
I20260809 08:52:25.274334 17091 webserver.cc:460] Webserver started at http://127.8.154.3:43579/ using document root <none> and password file <none>
I20260809 08:52:25.274605 17091 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:25.274654 17091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:25.274819 17091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:25.275964 17091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-2/data/instance:
uuid: "823ddfa425164cc69e6c15f20552e2f3"
format_stamp: "Formatted at 2026-08-09 08:52:25 on dist-test-slave-t3q3"
server_key: "a676d579244c3344dad2f5fe6e0549f0"
server_key_iv: "6f89532c1f72ef6f619d6505aed0cf46"
server_key_version: "encryptionkey@0"
I20260809 08:52:25.276430 17091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance:
uuid: "823ddfa425164cc69e6c15f20552e2f3"
format_stamp: "Formatted at 2026-08-09 08:52:25 on dist-test-slave-t3q3"
server_key: "a676d579244c3344dad2f5fe6e0549f0"
server_key_iv: "6f89532c1f72ef6f619d6505aed0cf46"
server_key_version: "encryptionkey@0"
I20260809 08:52:25.277969 17091 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:52:25.278693 17119 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:25.278860 17091 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:52:25.278967 17091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-2/wal
uuid: "823ddfa425164cc69e6c15f20552e2f3"
format_stamp: "Formatted at 2026-08-09 08:52:25 on dist-test-slave-t3q3"
server_key: "a676d579244c3344dad2f5fe6e0549f0"
server_key_iv: "6f89532c1f72ef6f619d6505aed0cf46"
server_key_version: "encryptionkey@0"
I20260809 08:52:25.279044 17091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:25.282510 17091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:25.282809 17091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:25.282966 17091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:25.283175 17091 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:25.283491 17091 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:25.283547 17091 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:25.283586 17091 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:25.283613 17091 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:25.289573 17091 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.3:42433
I20260809 08:52:25.289619 17325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.3:42433 every 8 connection(s)
I20260809 08:52:25.289942 17091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
I20260809 08:52:25.292763 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 17091
I20260809 08:52:25.292893 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance
I20260809 08:52:25.293099 8808 external_mini_cluster.cc:1428] Setting key 8c5cff530e66196ef0f8dfd4442f63da
I20260809 08:52:25.293097 16638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:25.290627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:54999 (local address 127.8.154.62:36167)
0809 08:52:25.290676 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:25.290678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:25.290923 (+ 245us) server_negotiation.cc:407] Connection header received
0809 08:52:25.291445 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:25.291447 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:25.291453 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:25.291506 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:25.292790 (+ 1284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:25.292791 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:25.292792 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:52:25.292797 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:25.292813 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:52:25.292817 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:25.292884 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:25.293007 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 08:52:25.293025 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 08:52:25.293298 17329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:25.290602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:36167 (local address 127.8.154.3:54999)
0809 08:52:25.290796 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:25.290829 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:25.291323 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:25.291574 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:25.291578 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:25.291587 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:25.292606 (+ 1019us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:52:25.292607 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:25.292635 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:52:25.292665 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:25.292895 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:25.292901 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:25.292904 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:52:25.293002 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 08:52:25.293068 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":78,"threads_started":1}
I20260809 08:52:25.293735 17327 heartbeater.cc:347] Connected to a master server at 127.8.154.62:36167
I20260809 08:52:25.293843 17327 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:25.293991 17327 heartbeater.cc:511] Master 127.8.154.62:36167 requested a full tablet report, sending...
I20260809 08:52:25.294405 16569 ts_manager.cc:194] Registered new tserver with Master: 823ddfa425164cc69e6c15f20552e2f3 (127.8.154.3:42433)
I20260809 08:52:25.305670 8808 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:52:25.308497 16638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:25.307154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41862 (local address 127.8.154.62:36167)
0809 08:52:25.307231 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:25.307233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:25.307365 (+ 132us) server_negotiation.cc:407] Connection header received
0809 08:52:25.307435 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:25.307436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:25.307442 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:25.307515 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:25.308247 (+ 732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:25.308248 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:25.308248 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:52:25.308252 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:25.308271 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:52:25.308275 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:25.308346 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:25.308399 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 08:52:25.308418 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 08:52:25.310386 16569 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41862:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:52:25.312016 16569 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:52:25.316411 17350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:25.314384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:40331 (local address 127.0.0.1:41876)
0809 08:52:25.314698 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:25.314710 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:25.314763 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:25.315331 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:25.315337 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:25.315345 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:25.315908 (+ 563us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:52:25.315909 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:25.315927 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:52:25.315949 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:25.316261 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:25.316269 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:25.316273 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:52:25.316348 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 08:52:25.316360 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":2,"thread_start_us":78,"threads_started":1}
I20260809 08:52:25.316497 17352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:25.314678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41876 (local address 127.8.154.2:40331)
0809 08:52:25.315173 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:25.315178 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:25.315188 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:52:25.315214 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:25.315216 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:25.315225 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:25.315581 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:25.316045 (+ 464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:25.316052 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:25.316053 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:52:25.316083 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:25.316119 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:52:25.316143 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:25.316244 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:25.316374 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 08:52:25.316392 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":144,"threads_started":1}
I20260809 08:52:25.316635 17347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:25.314384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:39951 (local address 127.0.0.1:48100)
0809 08:52:25.314543 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:25.314584 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:25.314670 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:25.315355 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:25.315356 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:25.315359 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:25.316084 (+ 725us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:52:25.316085 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:25.316093 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:52:25.316108 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:25.316521 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:25.316522 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:25.316522 (+ 0us) client_negotiation.cc:769] Sending connection context
0809 08:52:25.316551 (+ 29us) client_negotiation.cc:240] Negotiation successful
0809 08:52:25.316565 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":98,"threads_started":1}
I20260809 08:52:25.316934 17353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:25.314941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48100 (local address 127.8.154.1:39951)
0809 08:52:25.315191 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:25.315196 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:25.315208 (+ 12us) server_negotiation.cc:407] Connection header received
0809 08:52:25.315236 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:25.315239 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:25.315248 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:25.315605 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:25.316222 (+ 617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:25.316230 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:25.316231 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:52:25.316236 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:25.316271 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:52:25.316295 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:25.316811 (+ 516us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:25.316836 (+ 25us) server_negotiation.cc:299] Negotiation successful
0809 08:52:25.316853 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":98,"threads_started":1}
I20260809 08:52:25.317226 16968 tablet_service.cc:1467] Processing CreateTablet for tablet 21edba0c73a24e04b88e3c2ea0da8f21 (DEFAULT_TABLE table=test-table [id=b0e26a7c7d1d422db57afe560118f82a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:25.317389 17346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:25.314386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:42433 (local address 127.0.0.1:54940)
0809 08:52:25.314910 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:25.314918 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:25.315016 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:25.315554 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:25.315556 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:25.315558 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:25.316408 (+ 850us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:52:25.316409 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:25.316418 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:52:25.316429 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:25.317227 (+ 798us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:25.317228 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:25.317229 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 08:52:25.317292 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 08:52:25.317306 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"mutex_wait_us":3,"thread_start_us":101,"threads_started":1}
I20260809 08:52:25.317667 16968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21edba0c73a24e04b88e3c2ea0da8f21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:25.317687 16759 tablet_service.cc:1467] Processing CreateTablet for tablet 21edba0c73a24e04b88e3c2ea0da8f21 (DEFAULT_TABLE table=test-table [id=b0e26a7c7d1d422db57afe560118f82a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:25.318046 16759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21edba0c73a24e04b88e3c2ea0da8f21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:25.321381 17354 tablet_bootstrap.cc:492] T 21edba0c73a24e04b88e3c2ea0da8f21 P 54f26aa30dcf4c1dbab883cee04efc70: Bootstrap starting.
I20260809 08:52:25.321507 17351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:25.314688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54940 (local address 127.8.154.3:42433)
0809 08:52:25.314921 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:25.314926 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:25.315030 (+ 104us) server_negotiation.cc:407] Connection header received
0809 08:52:25.315086 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:25.315089 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:25.315102 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:25.315195 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:25.317070 (+ 1875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:25.317078 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:25.317078 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:52:25.317083 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:25.317118 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:52:25.317145 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:25.317227 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:25.321389 (+ 4162us) server_negotiation.cc:299] Negotiation successful
0809 08:52:25.321412 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":120,"threads_started":1}
I20260809 08:52:25.322170 17354 tablet_bootstrap.cc:654] T 21edba0c73a24e04b88e3c2ea0da8f21 P 54f26aa30dcf4c1dbab883cee04efc70: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:25.322237 17216 tablet_service.cc:1467] Processing CreateTablet for tablet 21edba0c73a24e04b88e3c2ea0da8f21 (DEFAULT_TABLE table=test-table [id=b0e26a7c7d1d422db57afe560118f82a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:25.322542 17354 log.cc:824] T 21edba0c73a24e04b88e3c2ea0da8f21 P 54f26aa30dcf4c1dbab883cee04efc70: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:25.322654 17216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21edba0c73a24e04b88e3c2ea0da8f21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:25.322935 17355 tablet_bootstrap.cc:492] T 21edba0c73a24e04b88e3c2ea0da8f21 P 8afe19e4c8a94686869407da039e36db: Bootstrap starting.
I20260809 08:52:25.323545 17355 tablet_bootstrap.cc:654] T 21edba0c73a24e04b88e3c2ea0da8f21 P 8afe19e4c8a94686869407da039e36db: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:25.323827 17355 log.cc:824] T 21edba0c73a24e04b88e3c2ea0da8f21 P 8afe19e4c8a94686869407da039e36db: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:25.324427 17354 tablet_bootstrap.cc:492] T 21edba0c73a24e04b88e3c2ea0da8f21 P 54f26aa30dcf4c1dbab883cee04efc70: No bootstrap required, opened a new log
I20260809 08:52:25.324514 17354 ts_tablet_manager.cc:1397] T 21edba0c73a24e04b88e3c2ea0da8f21 P 54f26aa30dcf4c1dbab883cee04efc70: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 08:52:25.324637 17355 tablet_bootstrap.cc:492] T 21edba0c73a24e04b88e3c2ea0da8f21 P 8afe19e4c8a94686869407da039e36db: No bootstrap required, opened a new log
I20260809 08:52:25.324719 17355 ts_tablet_manager.cc:1397] T 21edba0c73a24e04b88e3c2ea0da8f21 P 8afe19e4c8a94686869407da039e36db: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:52:25.325819 17359 tablet_bootstrap.cc:492] T 21edba0c73a24e04b88e3c2ea0da8f21 P 823ddfa425164cc69e6c15f20552e2f3: Bootstrap starting.
I20260809 08:52:25.326629 17359 tablet_bootstrap.cc:654] T 21edba0c73a24e04b88e3c2ea0da8f21 P 823ddfa425164cc69e6c15f20552e2f3: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:25.326843 17354 raft_consensus.cc:348] T 21edba0c73a24e04b88e3c2ea0da8f21 P 54f26aa30dcf4c1dbab883cee04efc70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afe19e4c8a94686869407da039e36db" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 40331 } } peers { permanent_uuid: "54f26aa30dcf4c1dbab883cee04efc70" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 39951 } } peers { permanent_uuid: "823ddfa425164cc69e6c15f20552e2f3" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 42433 } }
I20260809 08:52:25.326959 17359 log.cc:824] T 21edba0c73a24e04b88e3c2ea0da8f21 P 823ddfa425164cc69e6c15f20552e2f3: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:25.327019 17354 raft_consensus.cc:374] T 21edba0c73a24e04b88e3c2ea0da8f21 P 54f26aa30dcf4c1dbab883cee04efc70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:25.327056 17354 raft_consensus.cc:729] T 21edba0c73a24e04b88e3c2ea0da8f21 P 54f26aa30dcf4c1dbab883cee04efc70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54f26aa30dcf4c1dbab883cee04efc70, State: Initialized, Role: FOLLOWER
I20260809 08:52:25.327196 17354 consensus_queue.cc:260] T 21edba0c73a24e04b88e3c2ea0da8f21 P 54f26aa30dcf4c1dbab883cee04efc70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afe19e4c8a94686869407da039e36db" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 40331 } } peers { permanent_uuid: "54f26aa30dcf4c1dbab883cee04efc70" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 39951 } } peers { permanent_uuid: "823ddfa425164cc69e6c15f20552e2f3" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 42433 } }
I20260809 08:52:25.327471 17354 ts_tablet_manager.cc:1428] T 21edba0c73a24e04b88e3c2ea0da8f21 P 54f26aa30dcf4c1dbab883cee04efc70: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:52:25.327359 17355 raft_consensus.cc:348] T 21edba0c73a24e04b88e3c2ea0da8f21 P 8afe19e4c8a94686869407da039e36db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afe19e4c8a94686869407da039e36db" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 40331 } } peers { permanent_uuid: "54f26aa30dcf4c1dbab883cee04efc70" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 39951 } } peers { permanent_uuid: "823ddfa425164cc69e6c15f20552e2f3" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 42433 } }
I20260809 08:52:25.327512 16863 heartbeater.cc:503] Master 127.8.154.62:36167 was elected leader, sending a full tablet report...
I20260809 08:52:25.327519 17355 raft_consensus.cc:374] T 21edba0c73a24e04b88e3c2ea0da8f21 P 8afe19e4c8a94686869407da039e36db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:25.327560 17355 raft_consensus.cc:729] T 21edba0c73a24e04b88e3c2ea0da8f21 P 8afe19e4c8a94686869407da039e36db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8afe19e4c8a94686869407da039e36db, State: Initialized, Role: FOLLOWER
I20260809 08:52:25.327670 17355 consensus_queue.cc:260] T 21edba0c73a24e04b88e3c2ea0da8f21 P 8afe19e4c8a94686869407da039e36db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afe19e4c8a94686869407da039e36db" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 40331 } } peers { permanent_uuid: "54f26aa30dcf4c1dbab883cee04efc70" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 39951 } } peers { permanent_uuid: "823ddfa425164cc69e6c15f20552e2f3" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 42433 } }
I20260809 08:52:25.327952 17355 ts_tablet_manager.cc:1428] T 21edba0c73a24e04b88e3c2ea0da8f21 P 8afe19e4c8a94686869407da039e36db: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260809 08:52:25.327977 17087 heartbeater.cc:503] Master 127.8.154.62:36167 was elected leader, sending a full tablet report...
I20260809 08:52:25.328336 17359 tablet_bootstrap.cc:492] T 21edba0c73a24e04b88e3c2ea0da8f21 P 823ddfa425164cc69e6c15f20552e2f3: No bootstrap required, opened a new log
I20260809 08:52:25.328402 17359 ts_tablet_manager.cc:1397] T 21edba0c73a24e04b88e3c2ea0da8f21 P 823ddfa425164cc69e6c15f20552e2f3: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 08:52:25.330479 17359 raft_consensus.cc:348] T 21edba0c73a24e04b88e3c2ea0da8f21 P 823ddfa425164cc69e6c15f20552e2f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afe19e4c8a94686869407da039e36db" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 40331 } } peers { permanent_uuid: "54f26aa30dcf4c1dbab883cee04efc70" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 39951 } } peers { permanent_uuid: "823ddfa425164cc69e6c15f20552e2f3" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 42433 } }
I20260809 08:52:25.330621 17359 raft_consensus.cc:374] T 21edba0c73a24e04b88e3c2ea0da8f21 P 823ddfa425164cc69e6c15f20552e2f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:25.330670 17359 raft_consensus.cc:729] T 21edba0c73a24e04b88e3c2ea0da8f21 P 823ddfa425164cc69e6c15f20552e2f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 823ddfa425164cc69e6c15f20552e2f3, State: Initialized, Role: FOLLOWER
I20260809 08:52:25.330785 17359 consensus_queue.cc:260] T 21edba0c73a24e04b88e3c2ea0da8f21 P 823ddfa425164cc69e6c15f20552e2f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afe19e4c8a94686869407da039e36db" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 40331 } } peers { permanent_uuid: "54f26aa30dcf4c1dbab883cee04efc70" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 39951 } } peers { permanent_uuid: "823ddfa425164cc69e6c15f20552e2f3" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 42433 } }
I20260809 08:52:25.331024 17359 ts_tablet_manager.cc:1428] T 21edba0c73a24e04b88e3c2ea0da8f21 P 823ddfa425164cc69e6c15f20552e2f3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:52:25.331072 17327 heartbeater.cc:503] Master 127.8.154.62:36167 was elected leader, sending a full tablet report...
W20260809 08:52:25.422930 17089 tablet.cc:2369] T 21edba0c73a24e04b88e3c2ea0da8f21 P 8afe19e4c8a94686869407da039e36db: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:52:25.439785 17363 raft_consensus.cc:482] T 21edba0c73a24e04b88e3c2ea0da8f21 P 54f26aa30dcf4c1dbab883cee04efc70 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:52:25.439927 17363 raft_consensus.cc:504] T 21edba0c73a24e04b88e3c2ea0da8f21 P 54f26aa30dcf4c1dbab883cee04efc70 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afe19e4c8a94686869407da039e36db" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 40331 } } peers { permanent_uuid: "54f26aa30dcf4c1dbab883cee04efc70" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 39951 } } peers { permanent_uuid: "823ddfa425164cc69e6c15f20552e2f3" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 42433 } }
I20260809 08:52:25.440397 17363 leader_election.cc:288] T 21edba0c73a24e04b88e3c2ea0da8f21 P 54f26aa30dcf4c1dbab883cee04efc70 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8afe19e4c8a94686869407da039e36db (127.8.154.2:40331), 823ddfa425164cc69e6c15f20552e2f3 (127.8.154.3:42433)
I20260809 08:52:25.441902 16865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:25.440620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:40331 (local address 127.8.154.1:44343)
0809 08:52:25.440683 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:25.440698 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:25.440762 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:25.440964 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:25.440965 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:25.440968 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:25.441463 (+ 495us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:52:25.441463 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:25.441473 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:52:25.441488 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:25.441716 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:25.441717 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:25.441718 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 08:52:25.441800 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 08:52:25.441815 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 08:52:25.441916 17352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:25.440693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:44343 (local address 127.8.154.2:40331)
0809 08:52:25.440746 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:25.440749 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:25.440762 (+ 13us) server_negotiation.cc:407] Connection header received
0809 08:52:25.440874 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:25.440876 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:25.440881 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:25.440945 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:25.441609 (+ 664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:25.441610 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:25.441610 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:52:25.441615 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:25.441636 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:52:25.441640 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:25.441710 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:25.441813 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 08:52:25.441825 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 08:52:25.442072 17368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:25.440624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:42433 (local address 127.8.154.1:43925)
0809 08:52:25.440885 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:25.440894 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:25.440969 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:25.441109 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:25.441110 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:25.441112 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:25.441656 (+ 544us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:52:25.441656 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:25.441664 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:52:25.441678 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:25.441889 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:25.441890 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:25.441891 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 08:52:25.441950 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 08:52:25.441964 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":64,"thread_start_us":102,"threads_started":1}
I20260809 08:52:25.442224 17351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:25.440768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:43925 (local address 127.8.154.3:42433)
0809 08:52:25.440829 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:25.440831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:25.440957 (+ 126us) server_negotiation.cc:407] Connection header received
0809 08:52:25.441036 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:25.441037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:25.441043 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:25.441094 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:25.441780 (+ 686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:25.441782 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:25.441782 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:52:25.441788 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:25.441810 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:52:25.441815 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:25.441873 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:25.442089 (+ 216us) server_negotiation.cc:299] Negotiation successful
0809 08:52:25.442107 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 08:52:25.442245 17002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21edba0c73a24e04b88e3c2ea0da8f21" candidate_uuid: "54f26aa30dcf4c1dbab883cee04efc70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8afe19e4c8a94686869407da039e36db" is_pre_election: true
I20260809 08:52:25.442406 17002 raft_consensus.cc:2454] T 21edba0c73a24e04b88e3c2ea0da8f21 P 8afe19e4c8a94686869407da039e36db [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54f26aa30dcf4c1dbab883cee04efc70 in term 0.
I20260809 08:52:25.442571 17242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21edba0c73a24e04b88e3c2ea0da8f21" candidate_uuid: "54f26aa30dcf4c1dbab883cee04efc70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "823ddfa425164cc69e6c15f20552e2f3" is_pre_election: true
I20260809 08:52:25.442674 16689 leader_election.cc:302] T 21edba0c73a24e04b88e3c2ea0da8f21 P 54f26aa30dcf4c1dbab883cee04efc70 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54f26aa30dcf4c1dbab883cee04efc70, 8afe19e4c8a94686869407da039e36db; no voters:
I20260809 08:52:25.442754 17242 raft_consensus.cc:2454] T 21edba0c73a24e04b88e3c2ea0da8f21 P 823ddfa425164cc69e6c15f20552e2f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54f26aa30dcf4c1dbab883cee04efc70 in term 0.
I20260809 08:52:25.442791 17363 raft_consensus.cc:2781] T 21edba0c73a24e04b88e3c2ea0da8f21 P 54f26aa30dcf4c1dbab883cee04efc70 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:52:25.442865 17363 raft_consensus.cc:482] T 21edba0c73a24e04b88e3c2ea0da8f21 P 54f26aa30dcf4c1dbab883cee04efc70 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:52:25.442889 17363 raft_consensus.cc:3037] T 21edba0c73a24e04b88e3c2ea0da8f21 P 54f26aa30dcf4c1dbab883cee04efc70 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:25.443701 17363 raft_consensus.cc:504] T 21edba0c73a24e04b88e3c2ea0da8f21 P 54f26aa30dcf4c1dbab883cee04efc70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afe19e4c8a94686869407da039e36db" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 40331 } } peers { permanent_uuid: "54f26aa30dcf4c1dbab883cee04efc70" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 39951 } } peers { permanent_uuid: "823ddfa425164cc69e6c15f20552e2f3" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 42433 } }
I20260809 08:52:25.443826 17363 leader_election.cc:288] T 21edba0c73a24e04b88e3c2ea0da8f21 P 54f26aa30dcf4c1dbab883cee04efc70 [CANDIDATE]: Term 1 election: Requested vote from peers 8afe19e4c8a94686869407da039e36db (127.8.154.2:40331), 823ddfa425164cc69e6c15f20552e2f3 (127.8.154.3:42433)
I20260809 08:52:25.443986 17002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21edba0c73a24e04b88e3c2ea0da8f21" candidate_uuid: "54f26aa30dcf4c1dbab883cee04efc70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8afe19e4c8a94686869407da039e36db"
I20260809 08:52:25.444031 17242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21edba0c73a24e04b88e3c2ea0da8f21" candidate_uuid: "54f26aa30dcf4c1dbab883cee04efc70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "823ddfa425164cc69e6c15f20552e2f3"
I20260809 08:52:25.444111 17002 raft_consensus.cc:3037] T 21edba0c73a24e04b88e3c2ea0da8f21 P 8afe19e4c8a94686869407da039e36db [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:25.444164 17242 raft_consensus.cc:3037] T 21edba0c73a24e04b88e3c2ea0da8f21 P 823ddfa425164cc69e6c15f20552e2f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:25.444820 17002 raft_consensus.cc:2454] T 21edba0c73a24e04b88e3c2ea0da8f21 P 8afe19e4c8a94686869407da039e36db [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54f26aa30dcf4c1dbab883cee04efc70 in term 1.
I20260809 08:52:25.444962 17242 raft_consensus.cc:2454] T 21edba0c73a24e04b88e3c2ea0da8f21 P 823ddfa425164cc69e6c15f20552e2f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54f26aa30dcf4c1dbab883cee04efc70 in term 1.
I20260809 08:52:25.445147 16689 leader_election.cc:302] T 21edba0c73a24e04b88e3c2ea0da8f21 P 54f26aa30dcf4c1dbab883cee04efc70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54f26aa30dcf4c1dbab883cee04efc70, 8afe19e4c8a94686869407da039e36db; no voters:
I20260809 08:52:25.445263 17363 raft_consensus.cc:2781] T 21edba0c73a24e04b88e3c2ea0da8f21 P 54f26aa30dcf4c1dbab883cee04efc70 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:52:25.445420 17363 raft_consensus.cc:686] T 21edba0c73a24e04b88e3c2ea0da8f21 P 54f26aa30dcf4c1dbab883cee04efc70 [term 1 LEADER]: Becoming Leader. State: Replica: 54f26aa30dcf4c1dbab883cee04efc70, State: Running, Role: LEADER
I20260809 08:52:25.445521 17363 consensus_queue.cc:237] T 21edba0c73a24e04b88e3c2ea0da8f21 P 54f26aa30dcf4c1dbab883cee04efc70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afe19e4c8a94686869407da039e36db" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 40331 } } peers { permanent_uuid: "54f26aa30dcf4c1dbab883cee04efc70" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 39951 } } peers { permanent_uuid: "823ddfa425164cc69e6c15f20552e2f3" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 42433 } }
I20260809 08:52:25.446480 16568 catalog_manager.cc:5495] T 21edba0c73a24e04b88e3c2ea0da8f21 P 54f26aa30dcf4c1dbab883cee04efc70 reported cstate change: term changed from 0 to 1, leader changed from <none> to 54f26aa30dcf4c1dbab883cee04efc70 (127.8.154.1). New cstate: current_term: 1 leader_uuid: "54f26aa30dcf4c1dbab883cee04efc70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8afe19e4c8a94686869407da039e36db" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 40331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "54f26aa30dcf4c1dbab883cee04efc70" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 39951 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "823ddfa425164cc69e6c15f20552e2f3" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 42433 } health_report { overall_health: UNKNOWN } } }
I20260809 08:52:25.461814 17353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:25.460742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48110 (local address 127.8.154.1:39951)
0809 08:52:25.460794 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:25.460796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:25.460817 (+ 21us) server_negotiation.cc:407] Connection header received
0809 08:52:25.460857 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:25.460858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:25.460872 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:25.460943 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:25.461585 (+ 642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:25.461586 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:25.461587 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:52:25.461591 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:25.461608 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:52:25.461612 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:25.461667 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:25.461722 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 08:52:25.461736 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 08:52:25.463685 17242 raft_consensus.cc:1261] T 21edba0c73a24e04b88e3c2ea0da8f21 P 823ddfa425164cc69e6c15f20552e2f3 [term 1 FOLLOWER]: Refusing update from remote peer 54f26aa30dcf4c1dbab883cee04efc70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:52:25.463699 17002 raft_consensus.cc:1261] T 21edba0c73a24e04b88e3c2ea0da8f21 P 8afe19e4c8a94686869407da039e36db [term 1 FOLLOWER]: Refusing update from remote peer 54f26aa30dcf4c1dbab883cee04efc70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:52:25.464095 17369 consensus_queue.cc:1035] T 21edba0c73a24e04b88e3c2ea0da8f21 P 54f26aa30dcf4c1dbab883cee04efc70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8afe19e4c8a94686869407da039e36db" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 40331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:52:25.464237 17363 consensus_queue.cc:1035] T 21edba0c73a24e04b88e3c2ea0da8f21 P 54f26aa30dcf4c1dbab883cee04efc70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "823ddfa425164cc69e6c15f20552e2f3" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 42433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:52:25.467623 17379 mvcc.cc:205] Tried to move back new op lower bound from 7316543674216337408 to 7316543674145603584. Current Snapshot: MvccSnapshot[applied={T|T < 7316543674216337408}]
I20260809 08:52:25.468815 17381 mvcc.cc:205] Tried to move back new op lower bound from 7316543674216337408 to 7316543674145603584. Current Snapshot: MvccSnapshot[applied={T|T < 7316543674216337408 or (T in {7316543674216337408})}]
I20260809 08:52:25.469089 16569 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:52:25.469189 16569 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41862:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:52:25.469998 16569 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f14b6a42e5b8485ba2e2bd88fadbf4c3: Sending DeleteTablet for 3 replicas of tablet 21edba0c73a24e04b88e3c2ea0da8f21
I20260809 08:52:25.470361 16759 tablet_service.cc:1514] Processing DeleteTablet for tablet 21edba0c73a24e04b88e3c2ea0da8f21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:25 UTC) from {username='slave'} at 127.0.0.1:48100
I20260809 08:52:25.470363 16968 tablet_service.cc:1514] Processing DeleteTablet for tablet 21edba0c73a24e04b88e3c2ea0da8f21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:25 UTC) from {username='slave'} at 127.0.0.1:41876
I20260809 08:52:25.470688 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 16666
I20260809 08:52:25.470727 17216 tablet_service.cc:1514] Processing DeleteTablet for tablet 21edba0c73a24e04b88e3c2ea0da8f21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:25 UTC) from {username='slave'} at 127.0.0.1:54940
I20260809 08:52:25.482318 17397 tablet_replica.cc:330] T 21edba0c73a24e04b88e3c2ea0da8f21 P 823ddfa425164cc69e6c15f20552e2f3: stopping tablet replica
I20260809 08:52:25.482506 17397 raft_consensus.cc:2229] T 21edba0c73a24e04b88e3c2ea0da8f21 P 823ddfa425164cc69e6c15f20552e2f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:52:25.482663 17397 raft_consensus.cc:2258] T 21edba0c73a24e04b88e3c2ea0da8f21 P 823ddfa425164cc69e6c15f20552e2f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:52:25.483888 17397 ts_tablet_manager.cc:1905] T 21edba0c73a24e04b88e3c2ea0da8f21 P 823ddfa425164cc69e6c15f20552e2f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:52:25.485635 17397 ts_tablet_manager.cc:1918] T 21edba0c73a24e04b88e3c2ea0da8f21 P 823ddfa425164cc69e6c15f20552e2f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:52:25.485754 17397 log.cc:1196] T 21edba0c73a24e04b88e3c2ea0da8f21 P 823ddfa425164cc69e6c15f20552e2f3: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-2/wal/wals/21edba0c73a24e04b88e3c2ea0da8f21
I20260809 08:52:25.485834 17395 tablet_replica.cc:330] T 21edba0c73a24e04b88e3c2ea0da8f21 P 8afe19e4c8a94686869407da039e36db: stopping tablet replica
I20260809 08:52:25.485975 17395 raft_consensus.cc:2229] T 21edba0c73a24e04b88e3c2ea0da8f21 P 8afe19e4c8a94686869407da039e36db [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:52:25.486056 17397 ts_tablet_manager.cc:1939] T 21edba0c73a24e04b88e3c2ea0da8f21 P 823ddfa425164cc69e6c15f20552e2f3: Deleting consensus metadata
I20260809 08:52:25.486125 17395 raft_consensus.cc:2258] T 21edba0c73a24e04b88e3c2ea0da8f21 P 8afe19e4c8a94686869407da039e36db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:52:25.486577 16549 catalog_manager.cc:4841] TS 823ddfa425164cc69e6c15f20552e2f3 (127.8.154.3:42433): tablet 21edba0c73a24e04b88e3c2ea0da8f21 (table test-table [id=b0e26a7c7d1d422db57afe560118f82a]) successfully deleted
I20260809 08:52:25.486644 17395 ts_tablet_manager.cc:1905] T 21edba0c73a24e04b88e3c2ea0da8f21 P 8afe19e4c8a94686869407da039e36db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:52:25.488003 17395 ts_tablet_manager.cc:1918] T 21edba0c73a24e04b88e3c2ea0da8f21 P 8afe19e4c8a94686869407da039e36db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:52:25.488119 17395 log.cc:1196] T 21edba0c73a24e04b88e3c2ea0da8f21 P 8afe19e4c8a94686869407da039e36db: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265519942966-8808-0/minicluster-data/ts-1/wal/wals/21edba0c73a24e04b88e3c2ea0da8f21
I20260809 08:52:25.488457 17395 ts_tablet_manager.cc:1939] T 21edba0c73a24e04b88e3c2ea0da8f21 P 8afe19e4c8a94686869407da039e36db: Deleting consensus metadata
W20260809 08:52:25.488529 16550 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.154.1:39951 (error 108)
I20260809 08:52:25.488929 16547 catalog_manager.cc:4841] TS 8afe19e4c8a94686869407da039e36db (127.8.154.2:40331): tablet 21edba0c73a24e04b88e3c2ea0da8f21 (table test-table [id=b0e26a7c7d1d422db57afe560118f82a]) successfully deleted
I20260809 08:52:25.489060 17346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:25.488922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:39951 (local address 127.0.0.1:48118)
0809 08:52:25.488971 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:25.489017 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.1:39951: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260809 08:52:25.489224 16550 catalog_manager.cc:4584] TS 54f26aa30dcf4c1dbab883cee04efc70 (127.8.154.1:39951): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 21edba0c73a24e04b88e3c2ea0da8f21: Network error: Client connection negotiation failed: client connection to 127.8.154.1:39951: connect: Connection refused (error 111)
I20260809 08:52:25.489252 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 16868
I20260809 08:52:25.510993 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 17091
I20260809 08:52:25.522820 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 16527
2026-08-09T08:52:25Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (777 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 08:52:25.550194 8808 mini_oidc.cc:152] Starting JWKS server
I20260809 08:52:25.551379 8808 webserver.cc:460] Webserver started at https://127.0.0.1:39621/ using document root <none> and password file <none>
I20260809 08:52:25.551512 8808 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 08:52:25.551702 8808 webserver.cc:460] Webserver started at http://127.0.0.1:40415/ using document root <none> and password file <none>
2026-08-09T08:52:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:52:25Z Disabled control of system clock
I20260809 08:52:25.561448 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:35207
--webserver_interface=127.8.154.62
--webserver_port=0
--builtin_ntp_servers=127.8.154.20:46019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:35207
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39621/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786265519942966-8808-0/testchainca.cert with env {}
W20260809 08:52:25.647485 17411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:25.647701 17411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:25.647724 17411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:25.648888 17411 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:52:25.648931 17411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:25.648943 17411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:25.648955 17411 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:52:25.648968 17411 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 08:52:25.648980 17411 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 08:52:25.648999 17411 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39621/jwks.json
I20260809 08:52:25.650542 17411 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:46019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:35207
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:35207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39621/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786265519942966-8808-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:25.650751 17411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:25.650914 17411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:25.656143 17417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:25.656294 17411 server_base.cc:1034] running on GCE node
W20260809 08:52:25.656154 17416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:25.656196 17419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:25.656716 17411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:25.657030 17411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:25.658200 17411 hybrid_clock.cc:648] HybridClock initialized: now 1786265545658184 us; error 46 us; skew 500 ppm
I20260809 08:52:25.659662 17411 webserver.cc:460] Webserver started at http://127.8.154.62:36193/ using document root <none> and password file <none>
I20260809 08:52:25.659922 17411 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:25.659978 17411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:25.660140 17411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:25.661334 17411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "c869e830ff3e45b8ac8a6e8fd2b88493"
format_stamp: "Formatted at 2026-08-09 08:52:25 on dist-test-slave-t3q3"
server_key: "2b4c056ba1d8fd98b643ff1e4d807eec"
server_key_iv: "8a00ba3bf646944cba2d832dcd762f51"
server_key_version: "encryptionkey@0"
I20260809 08:52:25.661770 17411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "c869e830ff3e45b8ac8a6e8fd2b88493"
format_stamp: "Formatted at 2026-08-09 08:52:25 on dist-test-slave-t3q3"
server_key: "2b4c056ba1d8fd98b643ff1e4d807eec"
server_key_iv: "8a00ba3bf646944cba2d832dcd762f51"
server_key_version: "encryptionkey@0"
I20260809 08:52:25.663314 17411 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 08:52:25.664255 17432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:25.664467 17411 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 08:52:25.664585 17411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "c869e830ff3e45b8ac8a6e8fd2b88493"
format_stamp: "Formatted at 2026-08-09 08:52:25 on dist-test-slave-t3q3"
server_key: "2b4c056ba1d8fd98b643ff1e4d807eec"
server_key_iv: "8a00ba3bf646944cba2d832dcd762f51"
server_key_version: "encryptionkey@0"
I20260809 08:52:25.664672 17411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:25.688930 17411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:25.689317 17411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:25.689486 17411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:25.693761 17411 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:35207
I20260809 08:52:25.693820 17517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:35207 every 8 connection(s)
I20260809 08:52:25.694217 17411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:52:25.695313 17522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:25.696512 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 17411
I20260809 08:52:25.696602 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:52:25.696808 8808 external_mini_cluster.cc:1428] Setting key 01662f418bf2d7b29c69d53467aa54c6
I20260809 08:52:25.698365 17522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c869e830ff3e45b8ac8a6e8fd2b88493: Bootstrap starting.
I20260809 08:52:25.699230 17522 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c869e830ff3e45b8ac8a6e8fd2b88493: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:25.699572 17522 log.cc:824] T 00000000000000000000000000000000 P c869e830ff3e45b8ac8a6e8fd2b88493: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:25.700424 17522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c869e830ff3e45b8ac8a6e8fd2b88493: No bootstrap required, opened a new log
I20260809 08:52:25.702791 17522 raft_consensus.cc:348] T 00000000000000000000000000000000 P c869e830ff3e45b8ac8a6e8fd2b88493 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c869e830ff3e45b8ac8a6e8fd2b88493" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 35207 } }
I20260809 08:52:25.702982 17522 raft_consensus.cc:374] T 00000000000000000000000000000000 P c869e830ff3e45b8ac8a6e8fd2b88493 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:25.703022 17522 raft_consensus.cc:729] T 00000000000000000000000000000000 P c869e830ff3e45b8ac8a6e8fd2b88493 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c869e830ff3e45b8ac8a6e8fd2b88493, State: Initialized, Role: FOLLOWER
I20260809 08:52:25.703169 17522 consensus_queue.cc:260] T 00000000000000000000000000000000 P c869e830ff3e45b8ac8a6e8fd2b88493 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c869e830ff3e45b8ac8a6e8fd2b88493" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 35207 } }
I20260809 08:52:25.703244 17526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:25.697372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38620 (local address 127.8.154.62:35207)
0809 08:52:25.697567 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:25.697574 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:25.697599 (+ 25us) server_negotiation.cc:407] Connection header received
0809 08:52:25.698080 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:25.698087 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:25.698151 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:25.698277 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:25.698572 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:25.700187 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:25.701036 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:25.701677 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:25.701814 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:25.702503 (+ 689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:25.702507 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:25.702507 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:52:25.702718 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:25.702762 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:52:25.702777 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:25.702852 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:25.702933 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 08:52:25.703020 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":93,"threads_started":1}
I20260809 08:52:25.703258 17522 raft_consensus.cc:388] T 00000000000000000000000000000000 P c869e830ff3e45b8ac8a6e8fd2b88493 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:52:25.703352 17522 raft_consensus.cc:482] T 00000000000000000000000000000000 P c869e830ff3e45b8ac8a6e8fd2b88493 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:52:25.703400 17522 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c869e830ff3e45b8ac8a6e8fd2b88493 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:25.704192 17522 raft_consensus.cc:504] T 00000000000000000000000000000000 P c869e830ff3e45b8ac8a6e8fd2b88493 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c869e830ff3e45b8ac8a6e8fd2b88493" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 35207 } }
I20260809 08:52:25.704309 17522 leader_election.cc:302] T 00000000000000000000000000000000 P c869e830ff3e45b8ac8a6e8fd2b88493 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c869e830ff3e45b8ac8a6e8fd2b88493; no voters:
I20260809 08:52:25.704317 8808 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:52:25.704479 17522 leader_election.cc:288] T 00000000000000000000000000000000 P c869e830ff3e45b8ac8a6e8fd2b88493 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:52:25.704515 17529 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c869e830ff3e45b8ac8a6e8fd2b88493 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:52:25.704665 17529 raft_consensus.cc:686] T 00000000000000000000000000000000 P c869e830ff3e45b8ac8a6e8fd2b88493 [term 1 LEADER]: Becoming Leader. State: Replica: c869e830ff3e45b8ac8a6e8fd2b88493, State: Running, Role: LEADER
I20260809 08:52:25.704772 17522 sys_catalog.cc:564] T 00000000000000000000000000000000 P c869e830ff3e45b8ac8a6e8fd2b88493 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:52:25.704752 17529 consensus_queue.cc:237] T 00000000000000000000000000000000 P c869e830ff3e45b8ac8a6e8fd2b88493 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c869e830ff3e45b8ac8a6e8fd2b88493" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 35207 } }
I20260809 08:52:25.705224 17531 sys_catalog.cc:455] T 00000000000000000000000000000000 P c869e830ff3e45b8ac8a6e8fd2b88493 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c869e830ff3e45b8ac8a6e8fd2b88493" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c869e830ff3e45b8ac8a6e8fd2b88493" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 35207 } } }
I20260809 08:52:25.705314 17531 sys_catalog.cc:458] T 00000000000000000000000000000000 P c869e830ff3e45b8ac8a6e8fd2b88493 [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:25.705463 17531 sys_catalog.cc:455] T 00000000000000000000000000000000 P c869e830ff3e45b8ac8a6e8fd2b88493 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c869e830ff3e45b8ac8a6e8fd2b88493. Latest consensus state: current_term: 1 leader_uuid: "c869e830ff3e45b8ac8a6e8fd2b88493" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c869e830ff3e45b8ac8a6e8fd2b88493" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 35207 } } }
I20260809 08:52:25.705534 17531 sys_catalog.cc:458] T 00000000000000000000000000000000 P c869e830ff3e45b8ac8a6e8fd2b88493 [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:25.705654 17546 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:52:25.706332 17546 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:52:25.707933 17546 catalog_manager.cc:1347] Generated new cluster ID: 2ab704c2e61942248249f1c809b3b7ad
I20260809 08:52:25.707986 17546 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:52:25.730985 17546 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:52:25.731768 17546 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:52:25.740425 17546 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c869e830ff3e45b8ac8a6e8fd2b88493: Generated new TSK 0
I20260809 08:52:25.740657 17546 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 08:52:25.775544 17573 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 08:52:25.775733 17573 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260809 08:52:25.776476 17572 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:25.753471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:35207 (local address 127.0.0.1:38634)
0809 08:52:25.753689 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:25.753701 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:25.753779 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:25.754027 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:25.754027 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:25.754030 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0809 08:52:25.754106 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:25.754108 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:25.754815 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:25.754816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:25.755745 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:25.755749 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:25.756179 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:25.756180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:25.756250 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:25.756273 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 08:52:25.775891 (+ 19618us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39621/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 08:52:25.776014 (+ 123us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.154.62:35207: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39621/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":114,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.8.154.62:35207: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39621/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 08:52:25.776954 17526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:25.753528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38634 (local address 127.8.154.62:35207)
0809 08:52:25.753599 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:25.753601 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:25.753775 (+ 174us) server_negotiation.cc:407] Connection header received
0809 08:52:25.753913 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:25.753914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:25.753926 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:25.753998 (+ 72us) server_negotiation.cc:226] Negotiated authn=JWT
0809 08:52:25.754179 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:25.754710 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:25.755834 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:25.756029 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:25.756173 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:25.756392 (+ 219us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 08:52:25.775702 (+ 19310us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39621/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 08:52:25.776759 (+ 1057us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38634: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39621/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 08:52:25.776993 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 17411
W20260809 08:52:25.777024 17526 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38634: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39621/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
2026-08-09T08:52:25Z chronyd exiting
I20260809 08:52:25.823355 8808 test_util.cc:175] -----------------------------------------------
I20260809 08:52:25.823473 8808 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786265519942966-8808-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (274 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 08:52:25.824476 8808 mini_oidc.cc:152] Starting JWKS server
I20260809 08:52:25.825240 8808 webserver.cc:460] Webserver started at https://127.0.0.1:43887/ using document root <none> and password file <none>
I20260809 08:52:25.825313 8808 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 08:52:25.825479 8808 webserver.cc:460] Webserver started at http://127.0.0.1:32781/ using document root <none> and password file <none>
2026-08-09T08:52:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:52:25Z Disabled control of system clock
I20260809 08:52:25.835098 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:44363
--webserver_interface=127.8.154.62
--webserver_port=0
--builtin_ntp_servers=127.8.154.20:45183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:44363
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43887/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786265519942966-8808-0/testchainca.cert with env {}
W20260809 08:52:25.917953 17581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:25.918172 17581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:25.918195 17581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:25.919447 17581 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:52:25.919489 17581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:25.919502 17581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:25.919515 17581 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:52:25.919528 17581 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 08:52:25.919574 17581 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 08:52:25.919592 17581 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43887/jwks.json
I20260809 08:52:25.921146 17581 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:45183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:44363
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:44363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43887/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786265519942966-8808-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:25.921401 17581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:25.921644 17581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:25.926961 17595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:25.926923 17590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:25.926975 17581 server_base.cc:1034] running on GCE node
W20260809 08:52:25.926939 17591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:25.927469 17581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:25.927800 17581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:25.928982 17581 hybrid_clock.cc:648] HybridClock initialized: now 1786265545928967 us; error 50 us; skew 500 ppm
I20260809 08:52:25.930384 17581 webserver.cc:460] Webserver started at http://127.8.154.62:42513/ using document root <none> and password file <none>
I20260809 08:52:25.930686 17581 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:25.930761 17581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:25.930910 17581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:25.932152 17581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "9a1d4af00b3b4576a4d8f80281ad4212"
format_stamp: "Formatted at 2026-08-09 08:52:25 on dist-test-slave-t3q3"
server_key: "d1ad35baa6239077098b730895087416"
server_key_iv: "6f0c0d9c9da4f6f430e6860d0976f6fa"
server_key_version: "encryptionkey@0"
I20260809 08:52:25.932622 17581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "9a1d4af00b3b4576a4d8f80281ad4212"
format_stamp: "Formatted at 2026-08-09 08:52:25 on dist-test-slave-t3q3"
server_key: "d1ad35baa6239077098b730895087416"
server_key_iv: "6f0c0d9c9da4f6f430e6860d0976f6fa"
server_key_version: "encryptionkey@0"
I20260809 08:52:25.934129 17581 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 08:52:25.934888 17602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:25.935073 17581 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:52:25.935185 17581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "9a1d4af00b3b4576a4d8f80281ad4212"
format_stamp: "Formatted at 2026-08-09 08:52:25 on dist-test-slave-t3q3"
server_key: "d1ad35baa6239077098b730895087416"
server_key_iv: "6f0c0d9c9da4f6f430e6860d0976f6fa"
server_key_version: "encryptionkey@0"
I20260809 08:52:25.935263 17581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:25.947274 17581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:25.947583 17581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:25.947739 17581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:25.951718 17581 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:44363
I20260809 08:52:25.951756 17688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:44363 every 8 connection(s)
I20260809 08:52:25.952217 17581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:52:25.953294 17689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:25.955598 17689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a1d4af00b3b4576a4d8f80281ad4212: Bootstrap starting.
I20260809 08:52:25.956288 17689 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9a1d4af00b3b4576a4d8f80281ad4212: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:25.956542 17689 log.cc:824] T 00000000000000000000000000000000 P 9a1d4af00b3b4576a4d8f80281ad4212: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:25.957278 17689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9a1d4af00b3b4576a4d8f80281ad4212: No bootstrap required, opened a new log
I20260809 08:52:25.958830 17689 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9a1d4af00b3b4576a4d8f80281ad4212 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a1d4af00b3b4576a4d8f80281ad4212" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 44363 } }
I20260809 08:52:25.958967 17689 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9a1d4af00b3b4576a4d8f80281ad4212 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:25.958990 17689 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9a1d4af00b3b4576a4d8f80281ad4212 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a1d4af00b3b4576a4d8f80281ad4212, State: Initialized, Role: FOLLOWER
I20260809 08:52:25.959132 17689 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9a1d4af00b3b4576a4d8f80281ad4212 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a1d4af00b3b4576a4d8f80281ad4212" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 44363 } }
I20260809 08:52:25.959216 17689 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9a1d4af00b3b4576a4d8f80281ad4212 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:52:25.959239 17689 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9a1d4af00b3b4576a4d8f80281ad4212 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:52:25.959266 17689 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9a1d4af00b3b4576a4d8f80281ad4212 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:25.959797 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 17581
I20260809 08:52:25.959920 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:52:25.959889 17689 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9a1d4af00b3b4576a4d8f80281ad4212 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a1d4af00b3b4576a4d8f80281ad4212" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 44363 } }
I20260809 08:52:25.959996 17689 leader_election.cc:302] T 00000000000000000000000000000000 P 9a1d4af00b3b4576a4d8f80281ad4212 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9a1d4af00b3b4576a4d8f80281ad4212; no voters:
I20260809 08:52:25.960179 8808 external_mini_cluster.cc:1428] Setting key fb871f908c09ba5d23a15922bf225e3c
I20260809 08:52:25.960209 17689 leader_election.cc:288] T 00000000000000000000000000000000 P 9a1d4af00b3b4576a4d8f80281ad4212 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:52:25.960282 17695 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9a1d4af00b3b4576a4d8f80281ad4212 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:52:25.960476 17689 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9a1d4af00b3b4576a4d8f80281ad4212 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:52:25.960817 17695 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9a1d4af00b3b4576a4d8f80281ad4212 [term 1 LEADER]: Becoming Leader. State: Replica: 9a1d4af00b3b4576a4d8f80281ad4212, State: Running, Role: LEADER
I20260809 08:52:25.960919 17695 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9a1d4af00b3b4576a4d8f80281ad4212 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a1d4af00b3b4576a4d8f80281ad4212" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 44363 } }
I20260809 08:52:25.961285 17695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a1d4af00b3b4576a4d8f80281ad4212 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9a1d4af00b3b4576a4d8f80281ad4212. Latest consensus state: current_term: 1 leader_uuid: "9a1d4af00b3b4576a4d8f80281ad4212" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a1d4af00b3b4576a4d8f80281ad4212" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 44363 } } }
I20260809 08:52:25.961372 17695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a1d4af00b3b4576a4d8f80281ad4212 [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:25.961391 17697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9a1d4af00b3b4576a4d8f80281ad4212 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9a1d4af00b3b4576a4d8f80281ad4212" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9a1d4af00b3b4576a4d8f80281ad4212" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 44363 } } }
I20260809 08:52:25.961444 17697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9a1d4af00b3b4576a4d8f80281ad4212 [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:25.961581 17711 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:52:25.962090 17711 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:52:25.964500 17711 catalog_manager.cc:1347] Generated new cluster ID: a7da7a36fd974990b75d4b99323d9a74
I20260809 08:52:25.964553 17711 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:52:25.966485 17700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:25.960701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45762 (local address 127.8.154.62:44363)
0809 08:52:25.961176 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:25.961187 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:25.961237 (+ 50us) server_negotiation.cc:407] Connection header received
0809 08:52:25.961367 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:25.961374 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:25.961442 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:25.961638 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:25.961936 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:25.963448 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:25.964329 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:25.964926 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:25.965041 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:25.965778 (+ 737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:25.965782 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:25.965782 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:52:25.965984 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:25.966024 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:52:25.966037 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:25.966099 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:25.966191 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 08:52:25.966276 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":79,"threads_started":1}
I20260809 08:52:25.967440 8808 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:52:25.970803 17711 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:52:25.971477 17711 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:52:25.983659 17711 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9a1d4af00b3b4576a4d8f80281ad4212: Generated new TSK 0
I20260809 08:52:25.983855 17711 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 08:52:26.011145 17737 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 08:52:26.011232 17737 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 08:52:26.011554 17700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:25.989369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45764 (local address 127.8.154.62:44363)
0809 08:52:25.989432 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:25.989434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:25.989562 (+ 128us) server_negotiation.cc:407] Connection header received
0809 08:52:25.989639 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:25.989641 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:25.989653 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:25.989747 (+ 94us) server_negotiation.cc:226] Negotiated authn=JWT
0809 08:52:25.989890 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:25.990377 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:25.991182 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:25.991410 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:25.991492 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:25.991676 (+ 184us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 08:52:26.011325 (+ 19649us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43887/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 08:52:26.011490 (+ 165us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45764: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43887/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":40}
W20260809 08:52:26.011628 17700 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45764: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43887/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260809 08:52:26.011659 17735 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:25.989287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:44363 (local address 127.0.0.1:45764)
0809 08:52:25.989493 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:25.989502 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:25.989579 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:25.989747 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:25.989748 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:25.989750 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 08:52:25.989811 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:25.989813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:25.990493 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:25.990494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:25.991092 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:25.991094 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:25.991497 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:25.991498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:25.991561 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:25.991565 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 08:52:26.011448 (+ 19883us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43887/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 08:52:26.011561 (+ 113us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.154.62:44363: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43887/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":107,"threads_started":1}
I20260809 08:52:26.011948 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 17581
2026-08-09T08:52:26Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (235 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 08:52:26.060388 8808 mini_oidc.cc:152] Starting JWKS server
I20260809 08:52:26.061139 8808 webserver.cc:460] Webserver started at https://127.0.0.1:43417/ using document root <none> and password file <none>
I20260809 08:52:26.061213 8808 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 08:52:26.061380 8808 webserver.cc:460] Webserver started at http://127.0.0.1:45775/ using document root <none> and password file <none>
2026-08-09T08:52:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:52:26Z Disabled control of system clock
I20260809 08:52:26.076700 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:38869
--webserver_interface=127.8.154.62
--webserver_port=0
--builtin_ntp_servers=127.8.154.20:44213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:38869
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43417/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 08:52:26.159742 17745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:26.159952 17745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:26.159974 17745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:26.161182 17745 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:52:26.161263 17745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:26.161285 17745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:26.161307 17745 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:52:26.161338 17745 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 08:52:26.161370 17745 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 08:52:26.161396 17745 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43417/jwks.json
I20260809 08:52:26.162788 17745 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:44213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:38869
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:38869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43417/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:26.163079 17745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:26.163376 17745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:26.168360 17762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:26.168377 17755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:26.168356 17758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:26.168694 17745 server_base.cc:1034] running on GCE node
I20260809 08:52:26.168984 17745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:26.169308 17745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:26.170478 17745 hybrid_clock.cc:648] HybridClock initialized: now 1786265546170446 us; error 49 us; skew 500 ppm
I20260809 08:52:26.172049 17745 webserver.cc:460] Webserver started at http://127.8.154.62:41135/ using document root <none> and password file <none>
I20260809 08:52:26.172363 17745 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:26.172427 17745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:26.172590 17745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:26.174216 17745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "cec7155ba588420cbf69baf1d8023e31"
format_stamp: "Formatted at 2026-08-09 08:52:26 on dist-test-slave-t3q3"
server_key: "813df8bf62acace530db9137e34afbcc"
server_key_iv: "90dc699c9be66fc5b51dd2c1ab74abe4"
server_key_version: "encryptionkey@0"
I20260809 08:52:26.174788 17745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "cec7155ba588420cbf69baf1d8023e31"
format_stamp: "Formatted at 2026-08-09 08:52:26 on dist-test-slave-t3q3"
server_key: "813df8bf62acace530db9137e34afbcc"
server_key_iv: "90dc699c9be66fc5b51dd2c1ab74abe4"
server_key_version: "encryptionkey@0"
I20260809 08:52:26.176970 17745 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.003s
I20260809 08:52:26.181438 17770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:26.181640 17745 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260809 08:52:26.181756 17745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "cec7155ba588420cbf69baf1d8023e31"
format_stamp: "Formatted at 2026-08-09 08:52:26 on dist-test-slave-t3q3"
server_key: "813df8bf62acace530db9137e34afbcc"
server_key_iv: "90dc699c9be66fc5b51dd2c1ab74abe4"
server_key_version: "encryptionkey@0"
I20260809 08:52:26.181852 17745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:26.205042 17745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:26.205415 17745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:26.205581 17745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:26.211054 17745 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:38869
I20260809 08:52:26.211131 17847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:38869 every 8 connection(s)
I20260809 08:52:26.211532 17745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:52:26.212459 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 17745
I20260809 08:52:26.212479 17849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:26.212590 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:52:26.212829 8808 external_mini_cluster.cc:1428] Setting key ab17d295488686cf1af1bb1dc960d1e6
I20260809 08:52:26.217845 17849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cec7155ba588420cbf69baf1d8023e31: Bootstrap starting.
I20260809 08:52:26.218762 17849 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cec7155ba588420cbf69baf1d8023e31: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:26.219127 17849 log.cc:824] T 00000000000000000000000000000000 P cec7155ba588420cbf69baf1d8023e31: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:26.220028 17849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cec7155ba588420cbf69baf1d8023e31: No bootstrap required, opened a new log
I20260809 08:52:26.220028 17853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:26.213447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43064 (local address 127.8.154.62:38869)
0809 08:52:26.213750 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:26.213758 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:26.213792 (+ 34us) server_negotiation.cc:407] Connection header received
0809 08:52:26.214727 (+ 935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:26.214737 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:26.214825 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:26.214995 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:26.215230 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:26.216725 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:26.217635 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:26.218321 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:26.218476 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:26.219187 (+ 711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:26.219192 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:26.219193 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:52:26.219410 (+ 217us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:26.219456 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:52:26.219472 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:26.219737 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:26.219777 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 08:52:26.219864 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":126,"threads_started":1}
I20260809 08:52:26.221099 8808 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:52:26.223052 17849 raft_consensus.cc:348] T 00000000000000000000000000000000 P cec7155ba588420cbf69baf1d8023e31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec7155ba588420cbf69baf1d8023e31" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 38869 } }
I20260809 08:52:26.223261 17849 raft_consensus.cc:374] T 00000000000000000000000000000000 P cec7155ba588420cbf69baf1d8023e31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:26.223318 17849 raft_consensus.cc:729] T 00000000000000000000000000000000 P cec7155ba588420cbf69baf1d8023e31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cec7155ba588420cbf69baf1d8023e31, State: Initialized, Role: FOLLOWER
I20260809 08:52:26.223461 17849 consensus_queue.cc:260] T 00000000000000000000000000000000 P cec7155ba588420cbf69baf1d8023e31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec7155ba588420cbf69baf1d8023e31" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 38869 } }
I20260809 08:52:26.223580 17849 raft_consensus.cc:388] T 00000000000000000000000000000000 P cec7155ba588420cbf69baf1d8023e31 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:52:26.223623 17849 raft_consensus.cc:482] T 00000000000000000000000000000000 P cec7155ba588420cbf69baf1d8023e31 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:52:26.223682 17849 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cec7155ba588420cbf69baf1d8023e31 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:26.224586 17849 raft_consensus.cc:504] T 00000000000000000000000000000000 P cec7155ba588420cbf69baf1d8023e31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec7155ba588420cbf69baf1d8023e31" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 38869 } }
I20260809 08:52:26.224722 17849 leader_election.cc:302] T 00000000000000000000000000000000 P cec7155ba588420cbf69baf1d8023e31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cec7155ba588420cbf69baf1d8023e31; no voters:
I20260809 08:52:26.224931 17849 leader_election.cc:288] T 00000000000000000000000000000000 P cec7155ba588420cbf69baf1d8023e31 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:52:26.224993 17856 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cec7155ba588420cbf69baf1d8023e31 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:52:26.225154 17856 raft_consensus.cc:686] T 00000000000000000000000000000000 P cec7155ba588420cbf69baf1d8023e31 [term 1 LEADER]: Becoming Leader. State: Replica: cec7155ba588420cbf69baf1d8023e31, State: Running, Role: LEADER
I20260809 08:52:26.225237 17849 sys_catalog.cc:564] T 00000000000000000000000000000000 P cec7155ba588420cbf69baf1d8023e31 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:52:26.225255 17856 consensus_queue.cc:237] T 00000000000000000000000000000000 P cec7155ba588420cbf69baf1d8023e31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec7155ba588420cbf69baf1d8023e31" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 38869 } }
I20260809 08:52:26.225687 17858 sys_catalog.cc:455] T 00000000000000000000000000000000 P cec7155ba588420cbf69baf1d8023e31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cec7155ba588420cbf69baf1d8023e31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec7155ba588420cbf69baf1d8023e31" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 38869 } } }
I20260809 08:52:26.225703 17862 sys_catalog.cc:455] T 00000000000000000000000000000000 P cec7155ba588420cbf69baf1d8023e31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cec7155ba588420cbf69baf1d8023e31. Latest consensus state: current_term: 1 leader_uuid: "cec7155ba588420cbf69baf1d8023e31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cec7155ba588420cbf69baf1d8023e31" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 38869 } } }
I20260809 08:52:26.225795 17858 sys_catalog.cc:458] T 00000000000000000000000000000000 P cec7155ba588420cbf69baf1d8023e31 [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:26.225816 17862 sys_catalog.cc:458] T 00000000000000000000000000000000 P cec7155ba588420cbf69baf1d8023e31 [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:26.226080 17870 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:52:26.226624 17870 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:52:26.228351 17870 catalog_manager.cc:1347] Generated new cluster ID: 47d59b1b8c7c46a08b5052fb90bddfe7
I20260809 08:52:26.228442 17870 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:52:26.246455 17870 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:52:26.247002 17870 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:52:26.254132 17870 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cec7155ba588420cbf69baf1d8023e31: Generated new TSK 0
I20260809 08:52:26.254341 17870 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:52:26.270661 17853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:26.266299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43074 (local address 127.8.154.62:38869)
0809 08:52:26.266348 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:26.266351 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:26.266458 (+ 107us) server_negotiation.cc:407] Connection header received
0809 08:52:26.266527 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:26.266529 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:26.266542 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:26.266619 (+ 77us) server_negotiation.cc:226] Negotiated authn=JWT
0809 08:52:26.266753 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:26.267884 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:26.268768 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:26.269119 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:26.269197 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:26.269367 (+ 170us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 08:52:26.270414 (+ 1047us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43417/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 08:52:26.270556 (+ 142us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43074: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43417/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":28}
W20260809 08:52:26.270728 17853 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43074: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43417/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 08:52:26.272620 17895 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:26.266213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:38869 (local address 127.0.0.1:43074)
0809 08:52:26.266375 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:26.266386 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:26.266449 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:26.266623 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:26.266624 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:26.266626 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 08:52:26.266679 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:26.266681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:26.267991 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:26.267992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:26.268685 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:26.268687 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:26.269210 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:26.269212 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:26.269258 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:26.269263 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 08:52:26.270524 (+ 1261us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43417/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 08:52:26.272526 (+ 2002us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.154.62:38869: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43417/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":1786,"thread_start_us":108,"threads_started":1}
I20260809 08:52:26.272847 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 17745
E20260809 08:52:26.291260 17896 webserver.cc:533] Webserver: sslize failed:
E20260809 08:52:26.291337 17896 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-09T08:52:26Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (260 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T08:52:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:52:26Z Disabled control of system clock
I20260809 08:52:26.330246 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:38721
--webserver_interface=127.8.154.62
--webserver_port=0
--builtin_ntp_servers=127.8.154.20:46141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:38721
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786265519942966-8808-0/testchainca.cert with env {}
W20260809 08:52:26.412173 17902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:26.412387 17902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:26.412410 17902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:26.413558 17902 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:52:26.413594 17902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:26.413606 17902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:26.413619 17902 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:52:26.413631 17902 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 08:52:26.413643 17902 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 08:52:26.413654 17902 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 08:52:26.415098 17902 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:46141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:38721
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:38721
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786265519942966-8808-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:26.415310 17902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:26.415475 17902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:26.420637 17917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:26.420691 17913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:26.420742 17912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:26.420866 17902 server_base.cc:1034] running on GCE node
I20260809 08:52:26.421242 17902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:26.421553 17902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:26.422739 17902 hybrid_clock.cc:648] HybridClock initialized: now 1786265546422718 us; error 37 us; skew 500 ppm
I20260809 08:52:26.424103 17902 webserver.cc:460] Webserver started at http://127.8.154.62:46441/ using document root <none> and password file <none>
I20260809 08:52:26.424386 17902 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:26.424458 17902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:26.424625 17902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:26.425792 17902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "5e70c426e0114da5997f0e1ea40c1368"
format_stamp: "Formatted at 2026-08-09 08:52:26 on dist-test-slave-t3q3"
server_key: "5803db62a2aeec05a2b3a89178b2e213"
server_key_iv: "69a97a9b18ccce48d7629bbd70038487"
server_key_version: "encryptionkey@0"
I20260809 08:52:26.426258 17902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "5e70c426e0114da5997f0e1ea40c1368"
format_stamp: "Formatted at 2026-08-09 08:52:26 on dist-test-slave-t3q3"
server_key: "5803db62a2aeec05a2b3a89178b2e213"
server_key_iv: "69a97a9b18ccce48d7629bbd70038487"
server_key_version: "encryptionkey@0"
I20260809 08:52:26.427759 17902 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:52:26.428653 17927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:26.428821 17902 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 08:52:26.428934 17902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "5e70c426e0114da5997f0e1ea40c1368"
format_stamp: "Formatted at 2026-08-09 08:52:26 on dist-test-slave-t3q3"
server_key: "5803db62a2aeec05a2b3a89178b2e213"
server_key_iv: "69a97a9b18ccce48d7629bbd70038487"
server_key_version: "encryptionkey@0"
I20260809 08:52:26.429015 17902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:26.447695 17902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:26.448051 17902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:26.448302 17902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:26.452482 17902 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:38721
I20260809 08:52:26.452591 18018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:38721 every 8 connection(s)
I20260809 08:52:26.452970 17902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:52:26.453972 18019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:26.454983 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 17902
I20260809 08:52:26.455104 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:52:26.455317 8808 external_mini_cluster.cc:1428] Setting key 7229f1488884c62f889982bb5298c839
I20260809 08:52:26.456846 18019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e70c426e0114da5997f0e1ea40c1368: Bootstrap starting.
I20260809 08:52:26.457523 18019 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e70c426e0114da5997f0e1ea40c1368: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:26.457826 18019 log.cc:824] T 00000000000000000000000000000000 P 5e70c426e0114da5997f0e1ea40c1368: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:26.458545 18019 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e70c426e0114da5997f0e1ea40c1368: No bootstrap required, opened a new log
I20260809 08:52:26.460510 18019 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e70c426e0114da5997f0e1ea40c1368 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e70c426e0114da5997f0e1ea40c1368" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 38721 } }
I20260809 08:52:26.460685 18019 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e70c426e0114da5997f0e1ea40c1368 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:26.460733 18019 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e70c426e0114da5997f0e1ea40c1368 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e70c426e0114da5997f0e1ea40c1368, State: Initialized, Role: FOLLOWER
I20260809 08:52:26.460892 18019 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e70c426e0114da5997f0e1ea40c1368 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e70c426e0114da5997f0e1ea40c1368" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 38721 } }
I20260809 08:52:26.460961 18025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:26.455861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38134 (local address 127.8.154.62:38721)
0809 08:52:26.456105 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:26.456115 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:26.456153 (+ 38us) server_negotiation.cc:407] Connection header received
0809 08:52:26.456652 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:26.456659 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:26.456719 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:26.456873 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:26.457065 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:26.458042 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:26.458907 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:26.459324 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:26.459464 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:26.460318 (+ 854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:26.460322 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:26.460322 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:52:26.460484 (+ 162us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:26.460517 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:52:26.460527 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:26.460613 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:26.460688 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 08:52:26.460771 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":83,"threads_started":1}
I20260809 08:52:26.460994 18019 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e70c426e0114da5997f0e1ea40c1368 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:52:26.461033 18019 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e70c426e0114da5997f0e1ea40c1368 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:52:26.461086 18019 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e70c426e0114da5997f0e1ea40c1368 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:26.461786 18019 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e70c426e0114da5997f0e1ea40c1368 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e70c426e0114da5997f0e1ea40c1368" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 38721 } }
I20260809 08:52:26.461921 18019 leader_election.cc:302] T 00000000000000000000000000000000 P 5e70c426e0114da5997f0e1ea40c1368 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e70c426e0114da5997f0e1ea40c1368; no voters:
I20260809 08:52:26.461993 8808 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:52:26.462085 18019 leader_election.cc:288] T 00000000000000000000000000000000 P 5e70c426e0114da5997f0e1ea40c1368 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:52:26.462255 18030 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e70c426e0114da5997f0e1ea40c1368 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:52:26.462316 18019 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e70c426e0114da5997f0e1ea40c1368 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:52:26.462404 18030 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e70c426e0114da5997f0e1ea40c1368 [term 1 LEADER]: Becoming Leader. State: Replica: 5e70c426e0114da5997f0e1ea40c1368, State: Running, Role: LEADER
I20260809 08:52:26.462499 18030 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e70c426e0114da5997f0e1ea40c1368 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e70c426e0114da5997f0e1ea40c1368" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 38721 } }
I20260809 08:52:26.462795 18030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e70c426e0114da5997f0e1ea40c1368 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e70c426e0114da5997f0e1ea40c1368. Latest consensus state: current_term: 1 leader_uuid: "5e70c426e0114da5997f0e1ea40c1368" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e70c426e0114da5997f0e1ea40c1368" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 38721 } } }
I20260809 08:52:26.462874 18030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e70c426e0114da5997f0e1ea40c1368 [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:26.462805 18031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e70c426e0114da5997f0e1ea40c1368 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e70c426e0114da5997f0e1ea40c1368" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e70c426e0114da5997f0e1ea40c1368" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 38721 } } }
I20260809 08:52:26.463080 18039 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:52:26.463261 18031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e70c426e0114da5997f0e1ea40c1368 [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:26.463822 18039 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:52:26.465487 18039 catalog_manager.cc:1347] Generated new cluster ID: 9ae982963b71461e976a71b81cf18b0e
I20260809 08:52:26.465538 18039 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:52:26.479468 18039 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:52:26.480196 18039 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:52:26.487568 18039 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e70c426e0114da5997f0e1ea40c1368: Generated new TSK 0
I20260809 08:52:26.487815 18039 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:52:26.961499 18025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:26.493198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38144 (local address 127.8.154.62:38721)
0809 08:52:26.493250 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:26.493252 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:26.493342 (+ 90us) server_negotiation.cc:407] Connection header received
0809 08:52:26.493431 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:26.493432 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:26.493445 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:26.493514 (+ 69us) server_negotiation.cc:226] Negotiated authn=JWT
0809 08:52:26.493662 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:26.494178 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:26.494986 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:26.495199 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:26.495278 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:26.495465 (+ 187us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 08:52:26.961175 (+465710us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 08:52:26.961406 (+ 231us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38144: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":27}
W20260809 08:52:26.961580 18067 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:26.493097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:38721 (local address 127.0.0.1:38144)
0809 08:52:26.493264 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:26.493274 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:26.493354 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:26.493518 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:26.493519 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:26.493521 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 08:52:26.493589 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:26.493592 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:26.494272 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:26.494273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:26.494884 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:26.494885 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:26.495283 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:26.495284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:26.495353 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:26.495358 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 08:52:26.961349 (+465991us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 08:52:26.961475 (+ 126us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.8.154.62:38721: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":91,"threads_started":1}
W20260809 08:52:26.961594 18025 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38144: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
I20260809 08:52:26.961920 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 17902
2026-08-09T08:52:26Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (669 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (93 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (91 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:52:27 dist-test-slave-t3q3 krb5kdc[18086](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:52:27 dist-test-slave-t3q3 krb5kdc[18086](info): set up 2 sockets
Aug 09 08:52:27 dist-test-slave-t3q3 krb5kdc[18086](info): commencing operation
krb5kdc: starting...
W20260809 08:52:28.466883 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.291s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260809 08:52:28.589722 8808 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.123s user 0.000s sys 0.002s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260809 08:52:28.725147 8808 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.135s user 0.000s sys 0.002s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260809 08:52:28.951993 8808 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.227s user 0.001s sys 0.001s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/krb5kdc/test-user.keytab.
W20260809 08:52:29.070609 8808 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.118s user 0.002s sys 0.000s
Aug 09 08:52:29 dist-test-slave-t3q3 krb5kdc[18086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:52:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:52:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
I20260809 08:52:29.114811 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:41259
--webserver_interface=127.8.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:37159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:41259
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:29.201876 18111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:29.202111 18111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:29.202134 18111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:29.203244 18111 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:52:29.203284 18111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:29.203298 18111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:29.203311 18111 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:52:29.203323 18111 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:52:29.204810 18111 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:37159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:41259
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:41259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:29.205052 18111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:29.205240 18111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:29.210515 18121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:29.210486 18131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:29.210548 18120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:29.210808 18111 server_base.cc:1034] running on GCE node
I20260809 08:52:29.211081 18111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:29.211438 18111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:29.212600 18111 hybrid_clock.cc:648] HybridClock initialized: now 1786265549212578 us; error 43 us; skew 500 ppm
Aug 09 08:52:29 dist-test-slave-t3q3 krb5kdc[18086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265549, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:29.219157 18111 init.cc:377] Logged in from keytab as kudu/127.8.154.62@KRBTEST.COM (short username kudu)
I20260809 08:52:29.219642 18111 webserver.cc:460] Webserver started at http://127.8.154.62:43813/ using document root <none> and password file <none>
I20260809 08:52:29.219839 18111 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:29.219879 18111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:29.219974 18111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:29.221118 18111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "4acdc51b9a9e47068e77931cd3b0b22f"
format_stamp: "Formatted at 2026-08-09 08:52:29 on dist-test-slave-t3q3"
server_key: "721b0b0697ebd8cfe81a7331de212042"
server_key_iv: "674b7244b9c7f5fc989d28fa61023784"
server_key_version: "encryptionkey@0"
I20260809 08:52:29.221441 18111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "4acdc51b9a9e47068e77931cd3b0b22f"
format_stamp: "Formatted at 2026-08-09 08:52:29 on dist-test-slave-t3q3"
server_key: "721b0b0697ebd8cfe81a7331de212042"
server_key_iv: "674b7244b9c7f5fc989d28fa61023784"
server_key_version: "encryptionkey@0"
I20260809 08:52:29.222755 18111 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:52:29.223541 18142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:29.223776 18111 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:29.223878 18111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "4acdc51b9a9e47068e77931cd3b0b22f"
format_stamp: "Formatted at 2026-08-09 08:52:29 on dist-test-slave-t3q3"
server_key: "721b0b0697ebd8cfe81a7331de212042"
server_key_iv: "674b7244b9c7f5fc989d28fa61023784"
server_key_version: "encryptionkey@0"
I20260809 08:52:29.223941 18111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:29.238168 18111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:29.240727 18111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:29.240887 18111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:29.245571 18111 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:41259
I20260809 08:52:29.245605 18221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:41259 every 8 connection(s)
I20260809 08:52:29.246059 18111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:52:29.247020 18223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:29.249428 18223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4acdc51b9a9e47068e77931cd3b0b22f: Bootstrap starting.
I20260809 08:52:29.250020 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 18111
I20260809 08:52:29.250134 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:52:29.250130 18223 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4acdc51b9a9e47068e77931cd3b0b22f: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:29.250383 8808 external_mini_cluster.cc:1428] Setting key 5831212cbdc1f2e5c230591bf40b0a68
I20260809 08:52:29.250473 18223 log.cc:824] T 00000000000000000000000000000000 P 4acdc51b9a9e47068e77931cd3b0b22f: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:29.251336 18223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4acdc51b9a9e47068e77931cd3b0b22f: No bootstrap required, opened a new log
I20260809 08:52:29.253942 18223 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4acdc51b9a9e47068e77931cd3b0b22f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4acdc51b9a9e47068e77931cd3b0b22f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 41259 } }
I20260809 08:52:29.254112 18223 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4acdc51b9a9e47068e77931cd3b0b22f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:29.254142 18223 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4acdc51b9a9e47068e77931cd3b0b22f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4acdc51b9a9e47068e77931cd3b0b22f, State: Initialized, Role: FOLLOWER
I20260809 08:52:29.254266 18223 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4acdc51b9a9e47068e77931cd3b0b22f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4acdc51b9a9e47068e77931cd3b0b22f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 41259 } }
I20260809 08:52:29.254360 18223 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4acdc51b9a9e47068e77931cd3b0b22f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:52:29.254416 18223 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4acdc51b9a9e47068e77931cd3b0b22f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:52:29.254467 18223 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4acdc51b9a9e47068e77931cd3b0b22f [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:29.255051 18223 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4acdc51b9a9e47068e77931cd3b0b22f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4acdc51b9a9e47068e77931cd3b0b22f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 41259 } }
I20260809 08:52:29.255160 18223 leader_election.cc:302] T 00000000000000000000000000000000 P 4acdc51b9a9e47068e77931cd3b0b22f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4acdc51b9a9e47068e77931cd3b0b22f; no voters:
I20260809 08:52:29.255321 18223 leader_election.cc:288] T 00000000000000000000000000000000 P 4acdc51b9a9e47068e77931cd3b0b22f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:52:29.255368 18230 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4acdc51b9a9e47068e77931cd3b0b22f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:52:29.255551 18230 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4acdc51b9a9e47068e77931cd3b0b22f [term 1 LEADER]: Becoming Leader. State: Replica: 4acdc51b9a9e47068e77931cd3b0b22f, State: Running, Role: LEADER
I20260809 08:52:29.255621 18223 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4acdc51b9a9e47068e77931cd3b0b22f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:52:29.255667 18230 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4acdc51b9a9e47068e77931cd3b0b22f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4acdc51b9a9e47068e77931cd3b0b22f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 41259 } }
I20260809 08:52:29.256111 18231 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4acdc51b9a9e47068e77931cd3b0b22f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4acdc51b9a9e47068e77931cd3b0b22f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4acdc51b9a9e47068e77931cd3b0b22f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 41259 } } }
I20260809 08:52:29.256227 18231 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4acdc51b9a9e47068e77931cd3b0b22f [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:29.256111 18235 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4acdc51b9a9e47068e77931cd3b0b22f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4acdc51b9a9e47068e77931cd3b0b22f. Latest consensus state: current_term: 1 leader_uuid: "4acdc51b9a9e47068e77931cd3b0b22f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4acdc51b9a9e47068e77931cd3b0b22f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 41259 } } }
I20260809 08:52:29.256423 18235 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4acdc51b9a9e47068e77931cd3b0b22f [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:29.257563 18241 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 09 08:52:29 dist-test-slave-t3q3 krb5kdc[18086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:29.258111 18241 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:52:29.260756 18241 catalog_manager.cc:1347] Generated new cluster ID: 55801b4d70b146b1bd7a5e35dc5d6eb4
I20260809 08:52:29.260797 18241 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:52:29.262478 18228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:29.250948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39528 (local address 127.8.154.62:41259)
0809 08:52:29.251133 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:29.251142 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:29.251176 (+ 34us) server_negotiation.cc:407] Connection header received
0809 08:52:29.251790 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:29.251800 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:29.251866 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:29.252018 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:29.252713 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:29.254204 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:29.255060 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:29.255729 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:29.256210 (+ 481us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:29.258697 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:29.258701 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:29.258701 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:29.258713 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:29.260641 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:29.261080 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:29.261083 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:29.261084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:29.261185 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:29.261432 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:29.261433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:29.261433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:29.261723 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:29.261858 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:29.262019 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 08:52:29.262257 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":85,"threads_started":1}
I20260809 08:52:29.292830 18241 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:52:29.293426 18241 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:52:29.297371 18241 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4acdc51b9a9e47068e77931cd3b0b22f: Generated new TSK 0
I20260809 08:52:29.297554 18241 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab.
Entry for principal kudu/127.8.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab.
I20260809 08:52:29.328140 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.1:0
--local_ip_for_outbound_sockets=127.8.154.1
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:41259
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:37159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:29.411492 18270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:29.411689 18270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:29.411713 18270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:29.412932 18270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:29.412972 18270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:29.413017 18270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.1
I20260809 08:52:29.414417 18270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:37159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:41259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.1
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:29.414594 18270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:29.414762 18270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 08:52:29.419871 18270 server_base.cc:1034] running on GCE node
W20260809 08:52:29.419816 18281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:29.419826 18279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:29.419800 18278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:29.420295 18270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:29.420550 18270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:29.421736 18270 hybrid_clock.cc:648] HybridClock initialized: now 1786265549421711 us; error 38 us; skew 500 ppm
Aug 09 08:52:29 dist-test-slave-t3q3 krb5kdc[18086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265549, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:29.424216 18270 init.cc:377] Logged in from keytab as kudu/127.8.154.1@KRBTEST.COM (short username kudu)
I20260809 08:52:29.424592 18270 webserver.cc:460] Webserver started at http://127.8.154.1:39925/ using document root <none> and password file <none>
I20260809 08:52:29.424772 18270 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:29.424813 18270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:29.424901 18270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:29.425956 18270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-0/data/instance:
uuid: "b40b97a6dbe84db9812aafdb46794988"
format_stamp: "Formatted at 2026-08-09 08:52:29 on dist-test-slave-t3q3"
server_key: "a9549f07c5d77dbf6b2096507ac6a7dc"
server_key_iv: "354fe3f892d0d38a66e2a51dcf5203c7"
server_key_version: "encryptionkey@0"
I20260809 08:52:29.426254 18270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance:
uuid: "b40b97a6dbe84db9812aafdb46794988"
format_stamp: "Formatted at 2026-08-09 08:52:29 on dist-test-slave-t3q3"
server_key: "a9549f07c5d77dbf6b2096507ac6a7dc"
server_key_iv: "354fe3f892d0d38a66e2a51dcf5203c7"
server_key_version: "encryptionkey@0"
I20260809 08:52:29.427563 18270 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 08:52:29.428362 18291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:29.428539 18270 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:29.428623 18270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-0/wal
uuid: "b40b97a6dbe84db9812aafdb46794988"
format_stamp: "Formatted at 2026-08-09 08:52:29 on dist-test-slave-t3q3"
server_key: "a9549f07c5d77dbf6b2096507ac6a7dc"
server_key_iv: "354fe3f892d0d38a66e2a51dcf5203c7"
server_key_version: "encryptionkey@0"
I20260809 08:52:29.428676 18270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:29.444624 18270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:29.446879 18270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:29.446997 18270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:29.447220 18270 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:29.447554 18270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:29.447592 18270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:29.447614 18270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:29.447662 18270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:29.453683 18270 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.1:38611
I20260809 08:52:29.453758 18484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.1:38611 every 8 connection(s)
I20260809 08:52:29.454088 18270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
Aug 09 08:52:29 dist-test-slave-t3q3 krb5kdc[18086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265549, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:29.463404 18228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:29.454815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:52971 (local address 127.8.154.62:41259)
0809 08:52:29.454913 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:29.454915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:29.455089 (+ 174us) server_negotiation.cc:407] Connection header received
0809 08:52:29.456109 (+ 1020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:29.456110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:29.456122 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:29.456184 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:29.456784 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:29.457301 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:29.458321 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:29.458555 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:29.458643 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:29.460802 (+ 2159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:29.460805 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:29.460805 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:29.460812 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:29.462020 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:29.462437 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:29.462438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:29.462439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:29.462498 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:29.462758 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:29.462759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:29.462759 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:29.462875 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:29.462947 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:29.463227 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 08:52:29.463311 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 08:52:29.463586 18488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:29.454766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:41259 (local address 127.8.154.1:52971)
0809 08:52:29.454967 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:29.455003 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:29.455972 (+ 969us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:29.456241 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:29.456244 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:29.456479 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:29.456686 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:29.456690 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:29.457380 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:29.457381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:29.458253 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:29.458255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:29.458656 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:29.458656 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:29.458759 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:29.459290 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:29.459298 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:29.460700 (+ 1402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:29.462107 (+ 1407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:29.462115 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:29.462117 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:29.462347 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:29.462570 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:29.462571 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:29.462571 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:29.462677 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:29.462939 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:29.462947 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:29.463086 (+ 139us) client_negotiation.cc:769] Sending connection context
0809 08:52:29.463191 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 08:52:29.463353 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":119,"threads_started":1}
I20260809 08:52:29.463843 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 18270
I20260809 08:52:29.463980 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance
I20260809 08:52:29.464167 18486 heartbeater.cc:347] Connected to a master server at 127.8.154.62:41259
I20260809 08:52:29.464272 18486 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:29.464274 8808 external_mini_cluster.cc:1428] Setting key 837eb52deffd5795410abc7a50ec8df6
I20260809 08:52:29.464478 18486 heartbeater.cc:511] Master 127.8.154.62:41259 requested a full tablet report, sending...
I20260809 08:52:29.465026 18163 ts_manager.cc:194] Registered new tserver with Master: b40b97a6dbe84db9812aafdb46794988 (127.8.154.1:38611)
I20260809 08:52:29.465652 18163 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.1@KRBTEST.COM'} at 127.8.154.1:52971
WARNING: no policy specified for kudu/127.8.154.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab.
Entry for principal kudu/127.8.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab.
I20260809 08:52:29.491000 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.2:0
--local_ip_for_outbound_sockets=127.8.154.2
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:41259
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:37159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:29.573776 18492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:29.573979 18492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:29.574002 18492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:29.575170 18492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:29.575215 18492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:29.575241 18492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.2
I20260809 08:52:29.576763 18492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:37159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:41259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.2
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:29.577015 18492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:29.577253 18492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:29.582477 18501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:29.582617 18492 server_base.cc:1034] running on GCE node
W20260809 08:52:29.582484 18502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:29.582494 18507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:29.582963 18492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:29.583216 18492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:29.584362 18492 hybrid_clock.cc:648] HybridClock initialized: now 1786265549584348 us; error 30 us; skew 500 ppm
Aug 09 08:52:29 dist-test-slave-t3q3 krb5kdc[18086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265549, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:29.586936 18492 init.cc:377] Logged in from keytab as kudu/127.8.154.2@KRBTEST.COM (short username kudu)
I20260809 08:52:29.587419 18492 webserver.cc:460] Webserver started at http://127.8.154.2:38893/ using document root <none> and password file <none>
I20260809 08:52:29.587602 18492 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:29.587643 18492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:29.587736 18492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:29.588804 18492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-1/data/instance:
uuid: "a34a3d77ef7a4e688e7b732f9abf5397"
format_stamp: "Formatted at 2026-08-09 08:52:29 on dist-test-slave-t3q3"
server_key: "4b433d82a09ef17aaf9664999bb36726"
server_key_iv: "5c3563ab9e4ba6bf1d46b8fd1cc96cd1"
server_key_version: "encryptionkey@0"
I20260809 08:52:29.589118 18492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance:
uuid: "a34a3d77ef7a4e688e7b732f9abf5397"
format_stamp: "Formatted at 2026-08-09 08:52:29 on dist-test-slave-t3q3"
server_key: "4b433d82a09ef17aaf9664999bb36726"
server_key_iv: "5c3563ab9e4ba6bf1d46b8fd1cc96cd1"
server_key_version: "encryptionkey@0"
I20260809 08:52:29.590490 18492 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:52:29.591248 18516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:29.591444 18492 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:29.591554 18492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-1/wal
uuid: "a34a3d77ef7a4e688e7b732f9abf5397"
format_stamp: "Formatted at 2026-08-09 08:52:29 on dist-test-slave-t3q3"
server_key: "4b433d82a09ef17aaf9664999bb36726"
server_key_iv: "5c3563ab9e4ba6bf1d46b8fd1cc96cd1"
server_key_version: "encryptionkey@0"
I20260809 08:52:29.591630 18492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:29.608919 18492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:29.610790 18492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:29.610962 18492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:29.611220 18492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:29.611544 18492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:29.611598 18492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:29.611632 18492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:29.611658 18492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:29.617799 18492 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.2:37801
I20260809 08:52:29.617863 18717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.2:37801 every 8 connection(s)
I20260809 08:52:29.618165 18492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
Aug 09 08:52:29 dist-test-slave-t3q3 krb5kdc[18086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265549, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:29.627005 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 18492
I20260809 08:52:29.627168 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance
I20260809 08:52:29.627337 18228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:29.618908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:54489 (local address 127.8.154.62:41259)
0809 08:52:29.618977 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:29.618978 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:29.619181 (+ 203us) server_negotiation.cc:407] Connection header received
0809 08:52:29.620164 (+ 983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:29.620166 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:29.620186 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:29.620261 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:29.620872 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:29.621404 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:29.622368 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:29.622560 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:29.622619 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:29.624659 (+ 2040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:29.624660 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:29.624660 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:29.624667 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:29.625884 (+ 1217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:29.626316 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:29.626317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:29.626317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:29.626384 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:29.626637 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:29.626638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:29.626638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:29.626772 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:29.626848 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:29.627161 (+ 313us) server_negotiation.cc:299] Negotiation successful
0809 08:52:29.627247 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 08:52:29.627458 8808 external_mini_cluster.cc:1428] Setting key 616917a88ab4db5085bc4eb3b1994d0c
I20260809 08:52:29.627503 18721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:29.618876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:41259 (local address 127.8.154.2:54489)
0809 08:52:29.619031 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:29.619072 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:29.619999 (+ 927us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:29.620321 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:29.620324 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:29.620572 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:29.620764 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:29.620769 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:29.621489 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:29.621490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:29.622287 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:29.622289 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:29.622638 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:29.622639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:29.622739 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:29.623168 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:29.623176 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:29.624576 (+ 1400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:29.625975 (+ 1399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:29.625983 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:29.625985 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:29.626230 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:29.626461 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:29.626463 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:29.626464 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:29.626566 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:29.626849 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:29.626857 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:29.626983 (+ 126us) client_negotiation.cc:769] Sending connection context
0809 08:52:29.627092 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 08:52:29.627266 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":82,"threads_started":1}
I20260809 08:52:29.627990 18718 heartbeater.cc:347] Connected to a master server at 127.8.154.62:41259
I20260809 08:52:29.628109 18718 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:29.628316 18718 heartbeater.cc:511] Master 127.8.154.62:41259 requested a full tablet report, sending...
I20260809 08:52:29.628713 18166 ts_manager.cc:194] Registered new tserver with Master: a34a3d77ef7a4e688e7b732f9abf5397 (127.8.154.2:37801)
I20260809 08:52:29.629098 18166 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.2@KRBTEST.COM'} at 127.8.154.2:54489
WARNING: no policy specified for kudu/127.8.154.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab.
Entry for principal kudu/127.8.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab.
I20260809 08:52:29.654429 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.3:0
--local_ip_for_outbound_sockets=127.8.154.3
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:41259
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:37159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:29.738880 18726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:29.739074 18726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:29.739097 18726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:29.740288 18726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:29.740332 18726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:29.740388 18726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.3
I20260809 08:52:29.741911 18726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:37159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:41259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.3
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:29.742200 18726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:29.742440 18726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:29.747746 18737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:29.747816 18735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:29.747785 18726 server_base.cc:1034] running on GCE node
W20260809 08:52:29.747748 18733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:29.748230 18726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:29.748523 18726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:29.749701 18726 hybrid_clock.cc:648] HybridClock initialized: now 1786265549749689 us; error 40 us; skew 500 ppm
Aug 09 08:52:29 dist-test-slave-t3q3 krb5kdc[18086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265549, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:29.752331 18726 init.cc:377] Logged in from keytab as kudu/127.8.154.3@KRBTEST.COM (short username kudu)
I20260809 08:52:29.752767 18726 webserver.cc:460] Webserver started at http://127.8.154.3:42217/ using document root <none> and password file <none>
I20260809 08:52:29.753046 18726 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:29.753254 18726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:29.753436 18726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:29.754557 18726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-2/data/instance:
uuid: "1c9addbf0f234ba5a79d3fdc7cfe91b7"
format_stamp: "Formatted at 2026-08-09 08:52:29 on dist-test-slave-t3q3"
server_key: "b2af23bb55cce17ee86366d96e04bf9f"
server_key_iv: "495f0a91a8ea27a2a11d357451d35c2e"
server_key_version: "encryptionkey@0"
I20260809 08:52:29.754920 18726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance:
uuid: "1c9addbf0f234ba5a79d3fdc7cfe91b7"
format_stamp: "Formatted at 2026-08-09 08:52:29 on dist-test-slave-t3q3"
server_key: "b2af23bb55cce17ee86366d96e04bf9f"
server_key_iv: "495f0a91a8ea27a2a11d357451d35c2e"
server_key_version: "encryptionkey@0"
I20260809 08:52:29.756567 18726 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260809 08:52:29.757434 18747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:29.757628 18726 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:29.757725 18726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-2/wal
uuid: "1c9addbf0f234ba5a79d3fdc7cfe91b7"
format_stamp: "Formatted at 2026-08-09 08:52:29 on dist-test-slave-t3q3"
server_key: "b2af23bb55cce17ee86366d96e04bf9f"
server_key_iv: "495f0a91a8ea27a2a11d357451d35c2e"
server_key_version: "encryptionkey@0"
I20260809 08:52:29.757800 18726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:29.765579 18726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:29.767279 18726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:29.767438 18726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:29.767649 18726 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:29.767949 18726 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:29.768005 18726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:29.768045 18726 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:29.768096 18726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:29.774072 18726 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.3:46627
I20260809 08:52:29.774143 18942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.3:46627 every 8 connection(s)
I20260809 08:52:29.774472 18726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
I20260809 08:52:29.780220 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 18726
I20260809 08:52:29.780300 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance
I20260809 08:52:29.780483 8808 external_mini_cluster.cc:1428] Setting key 988509917fe6cb54c2494cf3442e95b5
Aug 09 08:52:29 dist-test-slave-t3q3 krb5kdc[18086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265549, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:29.783833 18228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:29.775239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:60927 (local address 127.8.154.62:41259)
0809 08:52:29.775297 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:29.775299 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:29.775498 (+ 199us) server_negotiation.cc:407] Connection header received
0809 08:52:29.776549 (+ 1051us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:29.776551 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:29.776563 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:29.776626 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:29.777237 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:29.777750 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:29.778783 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:29.778991 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:29.779086 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:29.781451 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:29.781452 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:29.781453 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:29.781461 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:29.782680 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:29.783047 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:29.783047 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:29.783048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:29.783098 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:29.783314 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:29.783315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:29.783315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:29.783417 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:29.783468 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:29.783707 (+ 239us) server_negotiation.cc:299] Negotiation successful
0809 08:52:29.783768 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 08:52:29.784061 18946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:29.775218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:41259 (local address 127.8.154.3:60927)
0809 08:52:29.775384 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:29.775418 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:29.776440 (+ 1022us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:29.776696 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:29.776700 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:29.776954 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:29.777138 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:29.777142 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:29.777849 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:29.777850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:29.778670 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:29.778672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:29.779098 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:29.779099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:29.779228 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:29.779752 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:29.779760 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:29.781330 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:29.782755 (+ 1425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:29.782761 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:29.782763 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:29.782982 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:29.783159 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:29.783160 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:29.783160 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:29.783258 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:29.783473 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:29.783478 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:29.783590 (+ 112us) client_negotiation.cc:769] Sending connection context
0809 08:52:29.783684 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 08:52:29.783803 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":79,"threads_started":1}
I20260809 08:52:29.784541 18943 heartbeater.cc:347] Connected to a master server at 127.8.154.62:41259
I20260809 08:52:29.784636 18943 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:29.784835 18943 heartbeater.cc:511] Master 127.8.154.62:41259 requested a full tablet report, sending...
I20260809 08:52:29.785193 18166 ts_manager.cc:194] Registered new tserver with Master: 1c9addbf0f234ba5a79d3fdc7cfe91b7 (127.8.154.3:46627)
I20260809 08:52:29.785576 18166 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.3@KRBTEST.COM'} at 127.8.154.3:60927
I20260809 08:52:29.792373 8808 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/krb5kdc/test-admin.keytab.
Aug 09 08:52:29 dist-test-slave-t3q3 krb5kdc[18086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:29.808852 8808 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 08:52:29.810385 18954 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:29.809647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33898 (local address 127.8.154.1:38611)
0809 08:52:29.809818 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:29.809822 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:29.809833 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:52:29.809857 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:29.809859 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:29.809873 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:29.809942 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:29.810301 (+ 359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33898: BlockingRecv error: recv got EOF from 127.0.0.1:33898 (error 108)
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":93,"threads_started":1}
W20260809 08:52:29.810498 8808 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/krb5kdc/krb5cc)
Aug 09 08:52:29 dist-test-slave-t3q3 krb5kdc[18086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:29.811463 8808 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 08:52:29 dist-test-slave-t3q3 krb5kdc[18086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:29.812352 8808 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 08:52:29 dist-test-slave-t3q3 krb5kdc[18086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.154.1@KRBTEST.COM
I20260809 08:52:29.819985 18954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:29.812869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33914 (local address 127.8.154.1:38611)
0809 08:52:29.812923 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:29.812925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:29.812933 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:52:29.812984 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:29.812984 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:29.813003 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:29.813050 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:29.813424 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:29.814146 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:29.814855 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:29.815151 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:29.815205 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:29.817333 (+ 2128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:29.817340 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:29.817341 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:29.817346 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:29.818909 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:29.819258 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:29.819261 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:29.819261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:29.819316 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:29.819482 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:29.819483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:29.819483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:29.819603 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:29.819705 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:29.819770 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 08:52:29.819829 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260809 08:52:29.821934 18954 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:29.821331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33926 (local address 127.8.154.1:38611)
0809 08:52:29.821389 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:29.821391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:29.821418 (+ 27us) server_negotiation.cc:407] Connection header received
0809 08:52:29.821465 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:29.821465 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:29.821473 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:29.821518 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:29.821888 (+ 370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33926: BlockingRecv error: recv got EOF from 127.0.0.1:33926 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260809 08:52:29.822028 8808 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/krb5kdc/krb5cc' of type FILE
Aug 09 08:52:29 dist-test-slave-t3q3 krb5kdc[18086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:29.822944 8808 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 08:52:29 dist-test-slave-t3q3 krb5kdc[18086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:29.823796 8808 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 08:52:29 dist-test-slave-t3q3 krb5kdc[18086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.154.1@KRBTEST.COM
I20260809 08:52:29.830638 18954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:29.824341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33930 (local address 127.8.154.1:38611)
0809 08:52:29.824392 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:29.824395 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:29.824438 (+ 43us) server_negotiation.cc:407] Connection header received
0809 08:52:29.824509 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:29.824510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:29.824520 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:29.824590 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:29.824958 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:29.825489 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:29.826156 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:29.826369 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:29.826456 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:29.828418 (+ 1962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:29.828420 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:29.828420 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:29.828427 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:29.829535 (+ 1108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:29.829891 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:29.829892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:29.829892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:29.829952 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:29.830158 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:29.830159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:29.830159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:29.830256 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:29.830349 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:29.830461 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 08:52:29.830538 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260809 08:52:29.832211 18954 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:29.831617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33942 (local address 127.8.154.1:38611)
0809 08:52:29.831651 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:29.831653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:29.831699 (+ 46us) server_negotiation.cc:407] Connection header received
0809 08:52:29.831770 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:29.831771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:29.831779 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:29.831834 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:29.832156 (+ 322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33942: BlockingRecv error: recv got EOF from 127.0.0.1:33942 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 08:52:29.832290 8808 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265519942966-8808-0/krb5kdc/krb5cc' of type FILE
Aug 09 08:52:29 dist-test-slave-t3q3 krb5kdc[18086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:29.833225 8808 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 08:52:29 dist-test-slave-t3q3 krb5kdc[18086](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:29.834061 8808 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 08:52:29 dist-test-slave-t3q3 krb5kdc[18086](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.154.1@KRBTEST.COM
I20260809 08:52:29.840535 18954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:29.834537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33950 (local address 127.8.154.1:38611)
0809 08:52:29.834592 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:29.834593 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:29.834626 (+ 33us) server_negotiation.cc:407] Connection header received
0809 08:52:29.834671 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:29.834671 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:29.834678 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:29.834722 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:29.835032 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:29.835466 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:29.836181 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:29.836356 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:29.836412 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:29.838237 (+ 1825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:29.838238 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:29.838238 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:29.838244 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:29.839568 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:29.839856 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:29.839856 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:29.839857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:29.839906 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:29.840049 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:29.840050 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:29.840050 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:29.840191 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:29.840260 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:29.840353 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 08:52:29.840406 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 08:52:29.841216 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 18270
I20260809 08:52:29.852681 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 18492
I20260809 08:52:29.873265 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 18726
I20260809 08:52:29.883981 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 18111
2026-08-09T08:52:29Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2735 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:52:29 dist-test-slave-t3q3 krb5kdc[18975](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:52:29 dist-test-slave-t3q3 krb5kdc[18975](info): set up 2 sockets
Aug 09 08:52:29 dist-test-slave-t3q3 krb5kdc[18975](info): commencing operation
krb5kdc: starting...
W20260809 08:52:31.142774 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:52:31 dist-test-slave-t3q3 krb5kdc[18975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:52:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:52:31Z Disabled control of system clock
WARNING: no policy specified for oryx/127.8.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.62.keytab.
Entry for principal oryx/127.8.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.62.keytab.
I20260809 08:52:31.267257 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:45607
--webserver_interface=127.8.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.62.keytab
--principal=oryx/127.8.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:45337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:45607
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:31.349853 19000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:31.350114 19000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:31.350157 19000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:31.351289 19000 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:52:31.351361 19000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:31.351387 19000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:31.351415 19000 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:52:31.351445 19000 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:52:31.352959 19000 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:45337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:45607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.62.keytab
--principal=oryx/127.8.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:45607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19000
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:31.353266 19000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:31.353500 19000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:31.358711 19010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:31.358681 19011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:31.358685 19013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:31.358722 19000 server_base.cc:1034] running on GCE node
I20260809 08:52:31.359164 19000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:31.359484 19000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:31.360659 19000 hybrid_clock.cc:648] HybridClock initialized: now 1786265551360637 us; error 42 us; skew 500 ppm
Aug 09 08:52:31 dist-test-slave-t3q3 krb5kdc[18975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265551, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:31.363201 19000 init.cc:377] Logged in from keytab as oryx/127.8.154.62@KRBTEST.COM (short username oryx)
I20260809 08:52:31.363688 19000 webserver.cc:460] Webserver started at http://127.8.154.62:41183/ using document root <none> and password file <none>
I20260809 08:52:31.363942 19000 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:31.364015 19000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:31.364184 19000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:31.365252 19000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "a22f631ee17d42f2a60289505b9f1fe4"
format_stamp: "Formatted at 2026-08-09 08:52:31 on dist-test-slave-t3q3"
server_key: "de3b7bc78840afca4fb5768ee541db52"
server_key_iv: "1b1d9e0e0c22847e2eba59833ae2c933"
server_key_version: "encryptionkey@0"
I20260809 08:52:31.365607 19000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "a22f631ee17d42f2a60289505b9f1fe4"
format_stamp: "Formatted at 2026-08-09 08:52:31 on dist-test-slave-t3q3"
server_key: "de3b7bc78840afca4fb5768ee541db52"
server_key_iv: "1b1d9e0e0c22847e2eba59833ae2c933"
server_key_version: "encryptionkey@0"
I20260809 08:52:31.366976 19000 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:52:31.367727 19024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:31.367941 19000 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:52:31.368049 19000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "a22f631ee17d42f2a60289505b9f1fe4"
format_stamp: "Formatted at 2026-08-09 08:52:31 on dist-test-slave-t3q3"
server_key: "de3b7bc78840afca4fb5768ee541db52"
server_key_iv: "1b1d9e0e0c22847e2eba59833ae2c933"
server_key_version: "encryptionkey@0"
I20260809 08:52:31.368168 19000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:31.389989 19000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:31.392190 19000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:31.392369 19000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:31.396560 19000 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:45607
I20260809 08:52:31.396600 19116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:45607 every 8 connection(s)
I20260809 08:52:31.396996 19000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:52:31.398061 19118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:31.400532 19118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4: Bootstrap starting.
I20260809 08:52:31.401140 19118 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:31.401443 19118 log.cc:824] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:31.402163 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 19000
I20260809 08:52:31.402290 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:52:31.402386 19118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4: No bootstrap required, opened a new log
I20260809 08:52:31.402539 8808 external_mini_cluster.cc:1428] Setting key f41151eda26a85e0659f5ca4cf6bf178
I20260809 08:52:31.404439 19118 raft_consensus.cc:348] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } }
I20260809 08:52:31.404604 19118 raft_consensus.cc:374] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:31.404670 19118 raft_consensus.cc:729] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a22f631ee17d42f2a60289505b9f1fe4, State: Initialized, Role: FOLLOWER
I20260809 08:52:31.404834 19118 consensus_queue.cc:260] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } }
I20260809 08:52:31.404914 19118 raft_consensus.cc:388] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:52:31.404938 19118 raft_consensus.cc:482] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:52:31.405000 19118 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:31.405894 19118 raft_consensus.cc:504] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } }
I20260809 08:52:31.406032 19118 leader_election.cc:302] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a22f631ee17d42f2a60289505b9f1fe4; no voters:
I20260809 08:52:31.406220 19118 leader_election.cc:288] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:52:31.406244 19127 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:52:31.406358 19127 raft_consensus.cc:686] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [term 1 LEADER]: Becoming Leader. State: Replica: a22f631ee17d42f2a60289505b9f1fe4, State: Running, Role: LEADER
I20260809 08:52:31.406497 19118 sys_catalog.cc:564] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:52:31.406529 19127 consensus_queue.cc:237] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } }
I20260809 08:52:31.406965 19133 sys_catalog.cc:455] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a22f631ee17d42f2a60289505b9f1fe4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } }
I20260809 08:52:31.407003 19131 sys_catalog.cc:455] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a22f631ee17d42f2a60289505b9f1fe4. Latest consensus state: current_term: 1 leader_uuid: "a22f631ee17d42f2a60289505b9f1fe4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } }
I20260809 08:52:31.407066 19133 sys_catalog.cc:458] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:31.407104 19131 sys_catalog.cc:458] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:31.407290 19139 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:52:31.407941 19139 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:52:31.409878 19139 catalog_manager.cc:1347] Generated new cluster ID: ac149d96c0c84228a4d9a3175bba4c67
I20260809 08:52:31.409940 19139 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 08:52:31 dist-test-slave-t3q3 krb5kdc[18975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM
I20260809 08:52:31.415364 19126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:31.403152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54760 (local address 127.8.154.62:45607)
0809 08:52:31.403354 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:31.403364 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:31.403398 (+ 34us) server_negotiation.cc:407] Connection header received
0809 08:52:31.404428 (+ 1030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:31.404437 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:31.404503 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:31.404668 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:31.405458 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:31.406585 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:31.407678 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:31.408342 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:31.408467 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:31.411393 (+ 2926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:31.411397 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:31.411398 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:31.411410 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:31.413463 (+ 2053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:31.413846 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:31.413849 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:31.413849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:31.414008 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:31.414193 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:31.414194 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:31.414194 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:31.414480 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:31.414615 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:31.415033 (+ 418us) server_negotiation.cc:299] Negotiation successful
0809 08:52:31.415164 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":102,"threads_started":1}
I20260809 08:52:31.425005 19139 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:52:31.425578 19139 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:52:31.439746 19139 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4: Generated new TSK 0
I20260809 08:52:31.439965 19139 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.8.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.1.keytab.
Entry for principal oryx/127.8.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.1.keytab.
I20260809 08:52:31.476706 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.1:0
--local_ip_for_outbound_sockets=127.8.154.1
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:45607
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.1.keytab
--principal=oryx/127.8.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:45337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:31.564306 19157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:31.564514 19157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:31.564539 19157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:31.565701 19157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:31.565750 19157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:31.565778 19157 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.1
I20260809 08:52:31.567283 19157 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:45337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.1.keytab
--principal=oryx/127.8.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:45607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.1
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:31.567550 19157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:31.567826 19157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:31.573334 19172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:31.573338 19169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:31.573388 19176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:31.573685 19157 server_base.cc:1034] running on GCE node
I20260809 08:52:31.573908 19157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:31.574182 19157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:31.575359 19157 hybrid_clock.cc:648] HybridClock initialized: now 1786265551575355 us; error 38 us; skew 500 ppm
Aug 09 08:52:31 dist-test-slave-t3q3 krb5kdc[18975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265551, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:31.578138 19157 init.cc:377] Logged in from keytab as oryx/127.8.154.1@KRBTEST.COM (short username oryx)
I20260809 08:52:31.578593 19157 webserver.cc:460] Webserver started at http://127.8.154.1:35045/ using document root <none> and password file <none>
I20260809 08:52:31.578850 19157 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:31.578922 19157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:31.579083 19157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:31.580307 19157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-0/data/instance:
uuid: "ad6563f6630440b2afc6ba67e966bebf"
format_stamp: "Formatted at 2026-08-09 08:52:31 on dist-test-slave-t3q3"
server_key: "bc24f14331862f643254b0b14d7f4b8a"
server_key_iv: "bb502c2758e57f290bb659154d1bdefb"
server_key_version: "encryptionkey@0"
I20260809 08:52:31.580683 19157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance:
uuid: "ad6563f6630440b2afc6ba67e966bebf"
format_stamp: "Formatted at 2026-08-09 08:52:31 on dist-test-slave-t3q3"
server_key: "bc24f14331862f643254b0b14d7f4b8a"
server_key_iv: "bb502c2758e57f290bb659154d1bdefb"
server_key_version: "encryptionkey@0"
I20260809 08:52:31.582218 19157 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:31.583037 19192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:31.583235 19157 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:31.583350 19157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-0/wal
uuid: "ad6563f6630440b2afc6ba67e966bebf"
format_stamp: "Formatted at 2026-08-09 08:52:31 on dist-test-slave-t3q3"
server_key: "bc24f14331862f643254b0b14d7f4b8a"
server_key_iv: "bb502c2758e57f290bb659154d1bdefb"
server_key_version: "encryptionkey@0"
I20260809 08:52:31.583431 19157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:31.623174 19157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:31.625785 19157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:31.625959 19157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:31.626230 19157 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:31.626573 19157 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:31.626629 19157 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:31.626667 19157 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:31.626714 19157 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:31.632649 19157 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.1:44029
I20260809 08:52:31.632707 19389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.1:44029 every 8 connection(s)
I20260809 08:52:31.633077 19157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
Aug 09 08:52:31 dist-test-slave-t3q3 krb5kdc[18975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265551, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM
I20260809 08:52:31.642504 19126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:31.633879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:50337 (local address 127.8.154.62:45607)
0809 08:52:31.633946 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:31.633948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:31.634160 (+ 212us) server_negotiation.cc:407] Connection header received
0809 08:52:31.634691 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:31.634692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:31.634704 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:31.634751 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:31.635823 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:31.636409 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:31.637452 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:31.637670 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:31.637749 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:31.639906 (+ 2157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:31.639908 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:31.639908 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:31.639916 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:31.641244 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:31.641601 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:31.641602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:31.641602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:31.641654 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:31.641912 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:31.641913 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:31.641914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:31.642036 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:31.642113 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:31.642359 (+ 246us) server_negotiation.cc:299] Negotiation successful
0809 08:52:31.642436 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 08:52:31.642647 19392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:31.633858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45607 (local address 127.8.154.1:50337)
0809 08:52:31.634032 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:31.634068 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:31.634579 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:31.634830 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:31.634834 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:31.635510 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:31.635700 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:31.635705 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:31.636511 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:31.636512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:31.637342 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:31.637344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:31.637771 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:31.637772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:31.637904 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:31.638355 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:31.638363 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:31.639752 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:31.641319 (+ 1567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:31.641326 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:31.641327 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:31.641534 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:31.641713 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:31.641714 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:31.641714 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:31.641813 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:31.642114 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:31.642118 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:31.642242 (+ 124us) client_negotiation.cc:769] Sending connection context
0809 08:52:31.642323 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 08:52:31.642441 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":85,"threads_started":1}
I20260809 08:52:31.643216 19390 heartbeater.cc:347] Connected to a master server at 127.8.154.62:45607
I20260809 08:52:31.643328 19390 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:31.643391 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 19157
I20260809 08:52:31.643494 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance
I20260809 08:52:31.643543 19390 heartbeater.cc:511] Master 127.8.154.62:45607 requested a full tablet report, sending...
I20260809 08:52:31.643738 8808 external_mini_cluster.cc:1428] Setting key 960edb691bac054e187e9a9b675561a0
I20260809 08:52:31.644174 19049 ts_manager.cc:194] Registered new tserver with Master: ad6563f6630440b2afc6ba67e966bebf (127.8.154.1:44029)
I20260809 08:52:31.644870 19049 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.154.1@KRBTEST.COM'} at 127.8.154.1:50337
WARNING: no policy specified for oryx/127.8.154.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.154.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.2.keytab.
Entry for principal oryx/127.8.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.2.keytab.
I20260809 08:52:31.670696 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.2:0
--local_ip_for_outbound_sockets=127.8.154.2
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:45607
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.2.keytab
--principal=oryx/127.8.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:45337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:31.754549 19399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:31.754797 19399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:31.754837 19399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:31.755971 19399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:31.756085 19399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:31.756136 19399 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.2
I20260809 08:52:31.757630 19399 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:45337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.2.keytab
--principal=oryx/127.8.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:45607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.2
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:31.757901 19399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:31.758147 19399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:31.763298 19405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:31.763336 19406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:31.763394 19399 server_base.cc:1034] running on GCE node
W20260809 08:52:31.763317 19411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:31.763769 19399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:31.764011 19399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:31.765204 19399 hybrid_clock.cc:648] HybridClock initialized: now 1786265551765181 us; error 38 us; skew 500 ppm
Aug 09 08:52:31 dist-test-slave-t3q3 krb5kdc[18975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265551, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:31.767733 19399 init.cc:377] Logged in from keytab as oryx/127.8.154.2@KRBTEST.COM (short username oryx)
I20260809 08:52:31.768283 19399 webserver.cc:460] Webserver started at http://127.8.154.2:36863/ using document root <none> and password file <none>
I20260809 08:52:31.768473 19399 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:31.768513 19399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:31.768610 19399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:31.769615 19399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-1/data/instance:
uuid: "a7984b066bfa4a7cbea4fd04bd72c6d8"
format_stamp: "Formatted at 2026-08-09 08:52:31 on dist-test-slave-t3q3"
server_key: "7f21a1668823ca6658918f5182638add"
server_key_iv: "2484ec9112f6100b098ab68791cec17b"
server_key_version: "encryptionkey@0"
I20260809 08:52:31.769920 19399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance:
uuid: "a7984b066bfa4a7cbea4fd04bd72c6d8"
format_stamp: "Formatted at 2026-08-09 08:52:31 on dist-test-slave-t3q3"
server_key: "7f21a1668823ca6658918f5182638add"
server_key_iv: "2484ec9112f6100b098ab68791cec17b"
server_key_version: "encryptionkey@0"
I20260809 08:52:31.771359 19399 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:52:31.772151 19424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:31.772310 19399 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:52:31.772408 19399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-1/wal
uuid: "a7984b066bfa4a7cbea4fd04bd72c6d8"
format_stamp: "Formatted at 2026-08-09 08:52:31 on dist-test-slave-t3q3"
server_key: "7f21a1668823ca6658918f5182638add"
server_key_iv: "2484ec9112f6100b098ab68791cec17b"
server_key_version: "encryptionkey@0"
I20260809 08:52:31.772464 19399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:31.788924 19399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:31.790974 19399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:31.791090 19399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:31.791285 19399 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:31.791564 19399 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:31.791596 19399 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:31.791617 19399 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:31.791630 19399 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:31.797395 19399 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.2:32787
I20260809 08:52:31.797467 19597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.2:32787 every 8 connection(s)
I20260809 08:52:31.797804 19399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
Aug 09 08:52:31 dist-test-slave-t3q3 krb5kdc[18975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265551, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM
I20260809 08:52:31.806116 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 19399
I20260809 08:52:31.806293 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance
I20260809 08:52:31.806542 8808 external_mini_cluster.cc:1428] Setting key 550b8b4ca209e04c72bba57ba849a0f7
I20260809 08:52:31.807507 19126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:31.798583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:42729 (local address 127.8.154.62:45607)
0809 08:52:31.798668 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:31.798671 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:31.798857 (+ 186us) server_negotiation.cc:407] Connection header received
0809 08:52:31.799400 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:31.799402 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:31.799414 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:31.799469 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:31.800475 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:31.800991 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:31.802028 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:31.802228 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:31.802325 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:31.804611 (+ 2286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:31.804612 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:31.804613 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:31.804620 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:31.805872 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:31.806370 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:31.806372 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:31.806372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:31.806449 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:31.806772 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:31.806773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:31.806774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:31.806936 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:31.806999 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:31.807324 (+ 325us) server_negotiation.cc:299] Negotiation successful
0809 08:52:31.807417 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 08:52:31.807693 19602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:31.798536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45607 (local address 127.8.154.2:42729)
0809 08:52:31.798725 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:31.798769 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:31.799290 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:31.799545 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:31.799549 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:31.800172 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:31.800367 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:31.800372 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:31.801099 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:31.801099 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:31.801923 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:31.801925 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:31.802336 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:31.802337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:31.802467 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:31.803004 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:31.803013 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:31.804492 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:31.805984 (+ 1492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:31.805992 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:31.805994 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:31.806295 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:31.806534 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:31.806535 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:31.806535 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:31.806683 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:31.807001 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:31.807009 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:31.807166 (+ 157us) client_negotiation.cc:769] Sending connection context
0809 08:52:31.807278 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 08:52:31.807435 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":101,"threads_started":1}
I20260809 08:52:31.808097 19600 heartbeater.cc:347] Connected to a master server at 127.8.154.62:45607
I20260809 08:52:31.808189 19600 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:31.808391 19600 heartbeater.cc:511] Master 127.8.154.62:45607 requested a full tablet report, sending...
I20260809 08:52:31.808779 19049 ts_manager.cc:194] Registered new tserver with Master: a7984b066bfa4a7cbea4fd04bd72c6d8 (127.8.154.2:32787)
I20260809 08:52:31.809175 19049 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.154.2@KRBTEST.COM'} at 127.8.154.2:42729
WARNING: no policy specified for oryx/127.8.154.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.154.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.3.keytab.
Entry for principal oryx/127.8.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.3.keytab.
I20260809 08:52:31.834643 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.3:0
--local_ip_for_outbound_sockets=127.8.154.3
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:45607
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.3.keytab
--principal=oryx/127.8.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:45337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:31.920486 19606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:31.920725 19606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:31.920768 19606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:31.921923 19606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:31.921994 19606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:31.922047 19606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.3
I20260809 08:52:31.923516 19606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:45337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.3.keytab
--principal=oryx/127.8.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:45607
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.3
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:31.923780 19606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:31.924021 19606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 08:52:31.929369 19606 server_base.cc:1034] running on GCE node
W20260809 08:52:31.929332 19618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:31.929339 19612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:31.929339 19611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:31.929791 19606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:31.930079 19606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:31.931244 19606 hybrid_clock.cc:648] HybridClock initialized: now 1786265551931233 us; error 29 us; skew 500 ppm
Aug 09 08:52:31 dist-test-slave-t3q3 krb5kdc[18975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265551, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:31.933844 19606 init.cc:377] Logged in from keytab as oryx/127.8.154.3@KRBTEST.COM (short username oryx)
I20260809 08:52:31.934325 19606 webserver.cc:460] Webserver started at http://127.8.154.3:44923/ using document root <none> and password file <none>
I20260809 08:52:31.934557 19606 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:31.934633 19606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:31.934793 19606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:31.935964 19606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-2/data/instance:
uuid: "145b09aa53c74c73a021795b1fa0e19c"
format_stamp: "Formatted at 2026-08-09 08:52:31 on dist-test-slave-t3q3"
server_key: "71c44df4c62fc084b51d3638e15153b3"
server_key_iv: "e780284d8624427ed128f574f283b20b"
server_key_version: "encryptionkey@0"
I20260809 08:52:31.936407 19606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance:
uuid: "145b09aa53c74c73a021795b1fa0e19c"
format_stamp: "Formatted at 2026-08-09 08:52:31 on dist-test-slave-t3q3"
server_key: "71c44df4c62fc084b51d3638e15153b3"
server_key_iv: "e780284d8624427ed128f574f283b20b"
server_key_version: "encryptionkey@0"
I20260809 08:52:31.937865 19606 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:31.938637 19632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:31.938812 19606 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:52:31.938923 19606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-2/wal
uuid: "145b09aa53c74c73a021795b1fa0e19c"
format_stamp: "Formatted at 2026-08-09 08:52:31 on dist-test-slave-t3q3"
server_key: "71c44df4c62fc084b51d3638e15153b3"
server_key_iv: "e780284d8624427ed128f574f283b20b"
server_key_version: "encryptionkey@0"
I20260809 08:52:31.939003 19606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:31.954339 19606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:31.956664 19606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:31.956831 19606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:31.957082 19606 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:31.957403 19606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:31.957458 19606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:31.957504 19606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:31.957536 19606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:31.963896 19606 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.3:44751
I20260809 08:52:31.963955 19812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.3:44751 every 8 connection(s)
I20260809 08:52:31.964340 19606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
Aug 09 08:52:31 dist-test-slave-t3q3 krb5kdc[18975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265551, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.3@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM
I20260809 08:52:31.970410 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 19606
I20260809 08:52:31.970525 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance
I20260809 08:52:31.970741 8808 external_mini_cluster.cc:1428] Setting key 5bee67deec05eaae9f371c12cb7b7999
I20260809 08:52:31.973623 19126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:31.965037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:56607 (local address 127.8.154.62:45607)
0809 08:52:31.965085 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:31.965087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:31.965288 (+ 201us) server_negotiation.cc:407] Connection header received
0809 08:52:31.965774 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:31.965775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:31.965785 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:31.965841 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:31.966771 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:31.967268 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:31.968307 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:31.968490 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:31.968544 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:31.970841 (+ 2297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:31.970842 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:31.970843 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:31.970850 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:31.972416 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:31.972795 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:31.972796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:31.972796 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:31.972849 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:31.973093 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:31.973093 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:31.973094 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:31.973197 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:31.973262 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:31.973496 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 08:52:31.973559 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 08:52:31.973793 19815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:31.964979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45607 (local address 127.8.154.3:56607)
0809 08:52:31.965145 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:31.965189 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:31.965680 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:31.965896 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:31.965900 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:31.966481 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:31.966678 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:31.966682 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:31.967348 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:31.967349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:31.968230 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:31.968233 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:31.968581 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:31.968582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:31.968694 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:31.969157 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:31.969166 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:31.970746 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:31.972488 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:31.972495 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:31.972496 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:31.972729 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:31.972907 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:31.972908 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:31.972908 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:31.973035 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:31.973266 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:31.973271 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:31.973384 (+ 113us) client_negotiation.cc:769] Sending connection context
0809 08:52:31.973465 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 08:52:31.973582 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":94,"threads_started":1}
I20260809 08:52:31.974241 19813 heartbeater.cc:347] Connected to a master server at 127.8.154.62:45607
I20260809 08:52:31.974336 19813 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:31.974483 19813 heartbeater.cc:511] Master 127.8.154.62:45607 requested a full tablet report, sending...
I20260809 08:52:31.974812 19049 ts_manager.cc:194] Registered new tserver with Master: 145b09aa53c74c73a021795b1fa0e19c (127.8.154.3:44751)
I20260809 08:52:31.975201 19049 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.154.3@KRBTEST.COM'} at 127.8.154.3:56607
I20260809 08:52:31.982594 8808 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.8.154.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.154.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.154.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.61.keytab.
Entry for principal oryx/127.8.154.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.61.keytab.
I20260809 08:52:32.008932 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.61:41867
--webserver_interface=127.8.154.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.61.keytab
--principal=oryx/127.8.154.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:45337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:45607,127.8.154.61:41867
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:32.091264 19818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:32.091470 19818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:32.091491 19818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:32.092667 19818 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:52:32.092711 19818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:32.092727 19818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:32.092741 19818 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:52:32.092752 19818 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:52:32.094250 19818 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:45337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:45607,127.8.154.61:41867
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.61.keytab
--principal=oryx/127.8.154.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.61:41867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.154.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
W20260809 08:52:32.094352 19818 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.8.154.62:45607,127.8.154.61:41867'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 08:52:32.094534 19818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:32.094772 19818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:32.099851 19836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:32.099851 19830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:32.099877 19834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:32.100263 19818 server_base.cc:1034] running on GCE node
I20260809 08:52:32.100536 19818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:32.100837 19818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:32.101986 19818 hybrid_clock.cc:648] HybridClock initialized: now 1786265552101971 us; error 35 us; skew 500 ppm
Aug 09 08:52:32 dist-test-slave-t3q3 krb5kdc[18975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265552, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:32.104753 19818 init.cc:377] Logged in from keytab as oryx/127.8.154.61@KRBTEST.COM (short username oryx)
I20260809 08:52:32.105211 19818 webserver.cc:460] Webserver started at http://127.8.154.61:39481/ using document root <none> and password file <none>
I20260809 08:52:32.105471 19818 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:32.105517 19818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:32.105697 19818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:32.106966 19818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-1/data/instance:
uuid: "a9f86b88da034484ab3f19ad3b945ade"
format_stamp: "Formatted at 2026-08-09 08:52:32 on dist-test-slave-t3q3"
server_key: "ec4b4d5e2179f5eb888988668d2fe002"
server_key_iv: "7082d1e38329ab66359f196c41e45acd"
server_key_version: "encryptionkey@0"
I20260809 08:52:32.107398 19818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-1/wal/instance:
uuid: "a9f86b88da034484ab3f19ad3b945ade"
format_stamp: "Formatted at 2026-08-09 08:52:32 on dist-test-slave-t3q3"
server_key: "ec4b4d5e2179f5eb888988668d2fe002"
server_key_iv: "7082d1e38329ab66359f196c41e45acd"
server_key_version: "encryptionkey@0"
I20260809 08:52:32.108928 19818 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:52:32.109704 19850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:32.109910 19818 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:32.109992 19818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-1/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-1/wal
uuid: "a9f86b88da034484ab3f19ad3b945ade"
format_stamp: "Formatted at 2026-08-09 08:52:32 on dist-test-slave-t3q3"
server_key: "ec4b4d5e2179f5eb888988668d2fe002"
server_key_iv: "7082d1e38329ab66359f196c41e45acd"
server_key_version: "encryptionkey@0"
I20260809 08:52:32.110045 19818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:32.124267 19818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:32.126740 19818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:32.126866 19818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:32.130930 19818 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.61:41867
I20260809 08:52:32.130963 19932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.61:41867 every 8 connection(s)
I20260809 08:52:32.131369 19818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-1/data/info.pb
I20260809 08:52:32.132468 19933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:32.133725 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 19818
I20260809 08:52:32.134395 19933 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 08:52:32 dist-test-slave-t3q3 krb5kdc[18975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265552, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.61@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM
Aug 09 08:52:32 dist-test-slave-t3q3 krb5kdc[18975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.154.61@KRBTEST.COM
I20260809 08:52:32.143818 19126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.135141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54766 (local address 127.8.154.62:45607)
0809 08:52:32.135194 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:32.135196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:32.135370 (+ 174us) server_negotiation.cc:407] Connection header received
0809 08:52:32.135472 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:32.135473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:32.135486 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:32.135566 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:32.136309 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.136812 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.137746 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.137939 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.138043 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.140404 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:32.140405 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:32.140406 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:32.140413 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:32.142180 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.142650 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.142651 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.142652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.142733 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.143102 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.143103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.143104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.143286 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:32.143368 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:32.143659 (+ 291us) server_negotiation.cc:299] Negotiation successful
0809 08:52:32.143730 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 08:52:32.144038 19938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.135072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45607 (local address 127.0.0.1:54766)
0809 08:52:32.135246 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:32.135288 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:32.135401 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:32.135593 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:32.135596 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:32.135976 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:32.136213 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.136219 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.136923 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.136924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.137651 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.137653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.138055 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.138056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.138193 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.138726 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:32.138737 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:32.140316 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:32.142292 (+ 1976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:32.142298 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:32.142300 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:32.142559 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:32.142815 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:32.142816 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:32.142816 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:32.142995 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:32.143374 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:32.143381 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:32.143508 (+ 127us) client_negotiation.cc:769] Sending connection context
0809 08:52:32.143621 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 08:52:32.143784 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":86,"threads_started":1}
I20260809 08:52:32.144253 19937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.134417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41054 (local address 127.8.154.61:41867)
0809 08:52:32.134687 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:32.134698 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:32.134745 (+ 47us) server_negotiation.cc:407] Connection header received
0809 08:52:32.135348 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:32.135355 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:32.135403 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:32.135524 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:32.136344 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.137153 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.137911 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.138238 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.138319 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.140998 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:32.141007 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:32.141007 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:32.141014 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:32.142906 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.143311 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.143314 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.143314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.143377 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.143593 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.143594 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.143594 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.143759 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:32.143912 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:32.144032 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 08:52:32.144136 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":113,"threads_started":1}
I20260809 08:52:32.144840 19933 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.154.61" port: 41867 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 08:52:32 dist-test-slave-t3q3 krb5kdc[18975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265552, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.61@KRBTEST.COM for oryx/127.8.154.61@KRBTEST.COM
I20260809 08:52:32.150837 19938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.145104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.61:41867 (local address 127.0.0.1:41070)
0809 08:52:32.145154 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:32.145165 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:32.145214 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:32.145327 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:32.145327 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:32.145436 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:32.145466 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.145467 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.146076 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.146076 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.146631 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.146632 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.146983 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.146984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.147067 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.147501 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:32.147505 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:32.148686 (+ 1181us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:32.149979 (+ 1293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:32.149980 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:32.149980 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:32.150169 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:32.150349 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:32.150350 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:32.150350 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:32.150417 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:32.150618 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:32.150619 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:32.150653 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 08:52:32.150704 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 08:52:32.150778 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 08:52:32.150957 19937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.145140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41070 (local address 127.8.154.61:41867)
0809 08:52:32.145174 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:32.145176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:32.145202 (+ 26us) server_negotiation.cc:407] Connection header received
0809 08:52:32.145265 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:32.145266 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:32.145273 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:32.145315 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:32.145522 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.145966 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.146709 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.146896 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.146954 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.148760 (+ 1806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:32.148761 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:32.148761 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:32.148768 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:32.149906 (+ 1138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.150242 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.150243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.150243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.150292 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.150471 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.150472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.150472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.150565 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:32.150612 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:32.150721 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 08:52:32.150822 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":41,"server-negotiator.queue_time_us":18}
I20260809 08:52:32.152375 19933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade: Bootstrap starting.
I20260809 08:52:32.153221 19933 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:32.153505 19933 log.cc:824] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:32.154209 19933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade: No bootstrap required, opened a new log
I20260809 08:52:32.155651 19933 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 41867 } }
I20260809 08:52:32.155803 19933 raft_consensus.cc:374] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:32.155870 19933 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9f86b88da034484ab3f19ad3b945ade, State: Initialized, Role: FOLLOWER
I20260809 08:52:32.156021 19933 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 41867 } }
I20260809 08:52:32.156363 19943 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 41867 } } }
I20260809 08:52:32.156404 19933 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:52:32.156456 19943 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [sys.catalog]: This master's current role is: FOLLOWER
W20260809 08:52:32.158210 19959 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:52:32.158273 19959 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:52:32.159902 19818 master_runner.cc:428] Detected that this master a9f86b88da034484ab3f19ad3b945ade is joining an existing cluster
I20260809 08:52:32.159992 19818 master_runner.cc:432] Initiating AddMaster RPC to add 127.8.154.61:41867
I20260809 08:52:32.168380 19126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.161096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54778 (local address 127.8.154.62:45607)
0809 08:52:32.161146 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:32.161149 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:32.161446 (+ 297us) server_negotiation.cc:407] Connection header received
0809 08:52:32.161519 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:32.161520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:32.161532 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:32.161591 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:32.162111 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.162603 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.163665 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.163928 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.163960 (+ 32us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.165295 (+ 1335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:32.165297 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:32.165297 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:32.165304 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:32.166886 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.167406 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.167408 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.167408 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.167475 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.167707 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.167708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.167708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.167859 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:32.167907 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:32.168216 (+ 309us) server_negotiation.cc:299] Negotiation successful
0809 08:52:32.168289 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 08:52:32.168392 19970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.161002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45607 (local address 127.0.0.1:54778)
0809 08:52:32.161368 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:32.161377 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:32.161458 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:32.161641 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:32.161641 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:32.161907 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:32.162008 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.162010 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.162681 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.162681 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.163568 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.163570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.164033 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.164033 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.164137 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.164629 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:32.164641 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:32.165190 (+ 549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:32.166989 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:32.166990 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:32.166991 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:32.167330 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:32.167565 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:32.167566 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:32.167566 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:32.167628 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:32.167968 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:32.167968 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:32.168028 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 08:52:32.168172 (+ 144us) client_negotiation.cc:240] Negotiation successful
0809 08:52:32.168281 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":98,"threads_started":1}
I20260809 08:52:32.168838 19937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.161282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41076 (local address 127.8.154.61:41867)
0809 08:52:32.161316 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:32.161318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:32.161643 (+ 325us) server_negotiation.cc:407] Connection header received
0809 08:52:32.161675 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:32.161677 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:32.161689 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:32.161774 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:32.162132 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.162851 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.163831 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.164159 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.164236 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.165294 (+ 1058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:32.165296 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:32.165296 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:32.165303 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:32.167455 (+ 2152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.167841 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.167842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.167842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.167908 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.168128 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.168129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.168129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.168259 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:32.168331 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:32.168522 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 08:52:32.168656 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":19}
I20260809 08:52:32.168905 19968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.161229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.61:41867 (local address 127.0.0.1:41076)
0809 08:52:32.161561 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:32.161571 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:32.161618 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:32.161791 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:32.161792 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:32.162003 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:32.162052 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.162054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.162940 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.162941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.163748 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.163750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.164245 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.164246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.164315 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.164780 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:32.164784 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:32.165191 (+ 407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:32.167535 (+ 2344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:32.167536 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:32.167537 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:32.167764 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:32.167984 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:32.167985 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:32.167985 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:32.168051 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:32.168355 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:32.168355 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:32.168391 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 08:52:32.168480 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 08:52:32.168592 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":3,"thread_start_us":72,"threads_started":1}
Aug 09 08:52:32 dist-test-slave-t3q3 krb5kdc[18975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265551, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.62@KRBTEST.COM for oryx/127.8.154.61@KRBTEST.COM
I20260809 08:52:32.178450 19937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.171265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41088 (local address 127.8.154.61:41867)
0809 08:52:32.171310 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:32.171312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:32.171477 (+ 165us) server_negotiation.cc:407] Connection header received
0809 08:52:32.171599 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:32.171600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:32.171610 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:32.171683 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:32.172142 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.172588 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.173365 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.173557 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.173643 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.175904 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:32.175906 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:32.175906 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:32.175913 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:32.177249 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.177630 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.177631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.177632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.177694 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.177905 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.177906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.177906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.178016 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:32.178085 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:32.178280 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 08:52:32.178358 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 08:52:32.178457 19971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.171181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.61:41867 (local address 127.0.0.1:41088)
0809 08:52:32.171369 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:32.171413 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:32.171503 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:32.171691 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:32.171697 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:32.171969 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:32.172016 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.172018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.172679 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.172680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.173277 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.173279 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.173666 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.173667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.173752 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.174204 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:32.174208 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:32.175811 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:32.177334 (+ 1523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:32.177342 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:32.177344 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:32.177552 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:32.177765 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:32.177765 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:32.177766 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:32.177834 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:32.178091 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:32.178099 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:32.178150 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 08:52:32.178251 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 08:52:32.178361 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":92,"threads_started":1}
I20260809 08:52:32.178995 19049 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.8.154.61:41867: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade"
member_type: NON_VOTER
last_known_addr {
host: "127.8.154.61"
port: 41867
}
attrs {
promote: true
}
}
dest_uuid: "a22f631ee17d42f2a60289505b9f1fe4"
cas_config_opid_index: -1
I20260809 08:52:32.179376 19049 consensus_queue.cc:237] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: NON_VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: true } }
I20260809 08:52:32.179862 19974 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER a9f86b88da034484ab3f19ad3b945ade (127.8.154.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: NON_VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: true } } }
I20260809 08:52:32.180203 19892 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:32.181023 19974 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.8.154.61:41867
I20260809 08:52:32.181216 19892 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [term 1 FOLLOWER]: Refusing update from remote peer a22f631ee17d42f2a60289505b9f1fe4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 08:52:32.181285 19972 sys_catalog.cc:455] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a22f631ee17d42f2a60289505b9f1fe4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: NON_VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: true } } }
I20260809 08:52:32.181355 19972 sys_catalog.cc:458] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:32.181517 19818 master.cc:537] Master@127.8.154.61:41867 shutting down...
I20260809 08:52:32.181758 19972 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: NON_VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 08:52:32.182575 19026 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260809 08:52:32.184376 19026 consensus_peers.cc:487] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 -> Peer a9f86b88da034484ab3f19ad3b945ade (127.8.154.61:41867): Couldn't send request to peer a9f86b88da034484ab3f19ad3b945ade. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 08:52:32.184508 19818 raft_consensus.cc:2229] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:52:32.184592 19818 raft_consensus.cc:2258] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:52:32.184643 19818 tablet_replica.cc:330] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade: stopping tablet replica
I20260809 08:52:32.197599 19818 master.cc:552] Master@127.8.154.61:41867 shutdown complete.
I20260809 08:52:32.197670 19818 master_runner.cc:305] Clearing existing system tablet
I20260809 08:52:32.197811 19818 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:32.197877 19818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:32.198515 19818 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:52:32.199285 19980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:32.199416 19818 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:32.199486 19818 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 08:52:32.199524 19818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-1/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-1/wal
uuid: "a9f86b88da034484ab3f19ad3b945ade"
format_stamp: "Formatted at 2026-08-09 08:52:32 on dist-test-slave-t3q3"
server_key: "ec4b4d5e2179f5eb888988668d2fe002"
server_key_iv: "7082d1e38329ab66359f196c41e45acd"
server_key_version: "encryptionkey@0"
I20260809 08:52:32.200335 19818 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:52:32.201762 19818 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 08:52:32.201823 19818 log.cc:1196] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 08:52:32.202034 19818 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade: Deleting consensus metadata
I20260809 08:52:32.202171 19818 master_runner.cc:315] Copying system tablet from 127.8.154.62:45607
I20260809 08:52:32.202836 19818 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade: tablet copy: Beginning tablet copy session from remote peer at address 127.8.154.62:45607
I20260809 08:52:32.209604 19126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.203258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54790 (local address 127.8.154.62:45607)
0809 08:52:32.203308 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:32.203311 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:32.203412 (+ 101us) server_negotiation.cc:407] Connection header received
0809 08:52:32.203457 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:32.203458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:32.203468 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:32.203510 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:32.203908 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.204488 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.205606 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.205795 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.205847 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.206941 (+ 1094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:32.206943 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:32.206943 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:32.206950 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:32.208469 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.208889 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.208890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.208890 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.208944 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.209159 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.209160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.209160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.209263 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:32.209315 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:32.209481 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 08:52:32.209540 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 08:52:32.209676 19992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.203162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45607 (local address 127.0.0.1:54790)
0809 08:52:32.203321 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:32.203333 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:32.203404 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:32.203532 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:32.203533 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:32.203757 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:32.203830 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.203833 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.204581 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.204583 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.205521 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.205524 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.205867 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.205868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.205940 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.206417 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:32.206422 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:32.206835 (+ 413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:32.208548 (+ 1713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:32.208549 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:32.208549 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:32.208799 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:32.209007 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:32.209007 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:32.209008 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:32.209086 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:32.209327 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:32.209327 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:32.209379 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 08:52:32.209454 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 08:52:32.209563 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":92,"threads_started":1}
I20260809 08:52:32.210388 19089 tablet_copy_service.cc:140] P a22f631ee17d42f2a60289505b9f1fe4: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer a9f86b88da034484ab3f19ad3b945ade ({username='oryx', principal='oryx/127.8.154.61@KRBTEST.COM'} at 127.0.0.1:54790)
I20260809 08:52:32.210565 19089 tablet_copy_service.cc:161] P a22f631ee17d42f2a60289505b9f1fe4: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer a9f86b88da034484ab3f19ad3b945ade at {username='oryx', principal='oryx/127.8.154.61@KRBTEST.COM'} at 127.0.0.1:54790: session id = a9f86b88da034484ab3f19ad3b945ade-00000000000000000000000000000000
I20260809 08:52:32.211510 19089 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 08:52:32.212213 19818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 08:52:32.213300 19818 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade: tablet copy: Starting download of 0 data blocks...
I20260809 08:52:32.213474 19818 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade: tablet copy: Starting download of 1 WAL segments...
I20260809 08:52:32.214483 19818 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 08:52:32.215256 19089 tablet_copy_service.cc:342] P a22f631ee17d42f2a60289505b9f1fe4: Request end of tablet copy session a9f86b88da034484ab3f19ad3b945ade-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.8.154.61@KRBTEST.COM'} at 127.0.0.1:54790
I20260809 08:52:32.215318 19089 tablet_copy_service.cc:433] P a22f631ee17d42f2a60289505b9f1fe4: ending tablet copy session a9f86b88da034484ab3f19ad3b945ade-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer a9f86b88da034484ab3f19ad3b945ade
I20260809 08:52:32.217840 19818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:32.218955 19994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:32.219069 19995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:32.219105 19999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:32.219262 19818 server_base.cc:1034] running on GCE node
I20260809 08:52:32.219352 19818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:32.219491 19818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:32.220620 19818 hybrid_clock.cc:648] HybridClock initialized: now 1786265552220615 us; error 35 us; skew 500 ppm
Aug 09 08:52:32 dist-test-slave-t3q3 krb5kdc[18975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265552, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:32.221944 19818 init.cc:377] Logged in from keytab as oryx/127.8.154.61@KRBTEST.COM (short username oryx)
I20260809 08:52:32.222196 19818 webserver.cc:460] Webserver started at http://127.8.154.61:33581/ using document root <none> and password file <none>
I20260809 08:52:32.222332 19818 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:32.222370 19818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:32.222977 19818 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:52:32.223531 20013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:32.223656 19818 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:52:32.223706 19818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-1/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-1/wal
uuid: "a9f86b88da034484ab3f19ad3b945ade"
format_stamp: "Formatted at 2026-08-09 08:52:32 on dist-test-slave-t3q3"
server_key: "ec4b4d5e2179f5eb888988668d2fe002"
server_key_iv: "7082d1e38329ab66359f196c41e45acd"
server_key_version: "encryptionkey@0"
I20260809 08:52:32.223781 19818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:32.239828 19818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:32.241611 19818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:32.245067 19818 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.61:41867
I20260809 08:52:32.245251 20090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.61:41867 every 8 connection(s)
I20260809 08:52:32.245474 19818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-1/data/info.pb
I20260809 08:52:32.246410 20091 sys_catalog.cc:263] Verifying existing consensus state
I20260809 08:52:32.246881 20091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade: Bootstrap starting.
I20260809 08:52:32.250073 20091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 08:52:32.250313 20091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade: Bootstrap complete.
I20260809 08:52:32.250527 20091 raft_consensus.cc:348] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: NON_VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: true } }
I20260809 08:52:32.250615 20091 raft_consensus.cc:729] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a9f86b88da034484ab3f19ad3b945ade, State: Initialized, Role: LEARNER
I20260809 08:52:32.250684 20091 consensus_queue.cc:260] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: NON_VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: true } }
I20260809 08:52:32.250862 20096 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: NON_VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: true } } }
I20260809 08:52:32.251084 20091 sys_catalog.cc:564] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:52:32.253314 20118 catalog_manager.cc:1259] Loaded cluster ID: ac149d96c0c84228a4d9a3175bba4c67
I20260809 08:52:32.253351 20118 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade: loading cluster ID for follower catalog manager: success
I20260809 08:52:32.254544 20118 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade: acquiring CA information for follower catalog manager: success
I20260809 08:52:32.254781 20118 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 08:52:32.256124 20096 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [sys.catalog]: This master's current role is: LEARNER
I20260809 08:52:32.257436 20093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.245849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41108 (local address 127.8.154.61:41867)
0809 08:52:32.245974 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:32.245976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:32.245984 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:52:32.246008 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:32.246009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:32.246019 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:32.246113 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:32.246800 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.247800 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.248873 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.249182 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.249265 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.251576 (+ 2311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:32.251578 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:32.251578 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:32.251587 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:32.256308 (+ 4721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.256657 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.256658 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.256659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.256728 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.256897 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.256898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.256899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.257036 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:32.257142 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:32.257240 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 08:52:32.257342 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":71,"threads_started":1}
WARNING: no policy specified for oryx/127.8.154.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.154.60@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.154.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.60.keytab.
Entry for principal oryx/127.8.154.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.60.keytab.
I20260809 08:52:32.289001 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.60:39223
--webserver_interface=127.8.154.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.60.keytab
--principal=oryx/127.8.154.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:45337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:45607,127.8.154.61:41867,127.8.154.60:39223
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:32.385795 20122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:32.385986 20122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:32.386008 20122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:32.387138 20122 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:52:32.387178 20122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:32.387192 20122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:32.387205 20122 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:52:32.387217 20122 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:52:32.388789 20122 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:45337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:45607,127.8.154.61:41867,127.8.154.60:39223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.60.keytab
--principal=oryx/127.8.154.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.60:39223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.154.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:32.388983 20122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:32.389153 20122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:32.394094 20135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:32.394101 20132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:32.394101 20129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:32.394462 20122 server_base.cc:1034] running on GCE node
I20260809 08:52:32.394711 20122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:32.395013 20122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:32.396195 20122 hybrid_clock.cc:648] HybridClock initialized: now 1786265552396194 us; error 41 us; skew 500 ppm
Aug 09 08:52:32 dist-test-slave-t3q3 krb5kdc[18975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265552, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:32.399037 20122 init.cc:377] Logged in from keytab as oryx/127.8.154.60@KRBTEST.COM (short username oryx)
I20260809 08:52:32.399487 20122 webserver.cc:460] Webserver started at http://127.8.154.60:46739/ using document root <none> and password file <none>
I20260809 08:52:32.399755 20122 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:32.399828 20122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:32.399984 20122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:32.401019 20122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-2/data/instance:
uuid: "88b956d3e2a24162ac5e1709d2832b3c"
format_stamp: "Formatted at 2026-08-09 08:52:32 on dist-test-slave-t3q3"
server_key: "18e61d593f0c346db983c3e1f796b7d3"
server_key_iv: "def61ff707c4346a01d5ea06fc4a9131"
server_key_version: "encryptionkey@0"
I20260809 08:52:32.401468 20122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-2/wal/instance:
uuid: "88b956d3e2a24162ac5e1709d2832b3c"
format_stamp: "Formatted at 2026-08-09 08:52:32 on dist-test-slave-t3q3"
server_key: "18e61d593f0c346db983c3e1f796b7d3"
server_key_iv: "def61ff707c4346a01d5ea06fc4a9131"
server_key_version: "encryptionkey@0"
I20260809 08:52:32.402983 20122 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:52:32.403708 20146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:32.403893 20122 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:52:32.404013 20122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-2/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-2/wal
uuid: "88b956d3e2a24162ac5e1709d2832b3c"
format_stamp: "Formatted at 2026-08-09 08:52:32 on dist-test-slave-t3q3"
server_key: "18e61d593f0c346db983c3e1f796b7d3"
server_key_iv: "def61ff707c4346a01d5ea06fc4a9131"
server_key_version: "encryptionkey@0"
I20260809 08:52:32.404122 20122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:32.421277 20122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:32.423290 20122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:32.423465 20122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:32.429105 20122 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.60:39223
I20260809 08:52:32.429140 20229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.60:39223 every 8 connection(s)
I20260809 08:52:32.429522 20122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-2/data/info.pb
I20260809 08:52:32.430642 20230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:32.432670 20230 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:52:32.435006 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 20122
Aug 09 08:52:32 dist-test-slave-t3q3 krb5kdc[18975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265552, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.60@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM
Aug 09 08:52:32 dist-test-slave-t3q3 krb5kdc[18975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.154.60@KRBTEST.COM
I20260809 08:52:32.443046 19126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.433449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54802 (local address 127.8.154.62:45607)
0809 08:52:32.433547 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:32.433550 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:32.433738 (+ 188us) server_negotiation.cc:407] Connection header received
0809 08:52:32.434179 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:32.434180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:32.434191 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:32.434248 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:32.434873 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.435437 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.436671 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.436878 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.436968 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.439595 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:32.439597 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:32.439598 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:32.439607 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:32.441130 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.441631 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.441633 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.441633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.441719 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.442080 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.442081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.442082 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.442239 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:32.442324 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:32.442742 (+ 418us) server_negotiation.cc:299] Negotiation successful
0809 08:52:32.442878 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 08:52:32.443071 20234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.433348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45607 (local address 127.0.0.1:54802)
0809 08:52:32.433608 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:32.433644 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:32.434091 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:32.434313 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:32.434316 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:32.434563 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:32.434755 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.434759 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.435546 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.435547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.436582 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.436585 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.436981 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.436982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.437115 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.437680 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:32.437690 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:32.439500 (+ 1810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:32.441219 (+ 1719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:32.441227 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:32.441230 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:32.441532 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:32.441812 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:32.441813 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:32.441814 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:32.441959 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:32.442360 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:32.442367 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:32.442530 (+ 163us) client_negotiation.cc:769] Sending connection context
0809 08:52:32.442647 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 08:52:32.442870 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":179,"threads_started":1}
I20260809 08:52:32.443722 20230 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.154.61" port: 41867 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:52:32.445508 20235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.435576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33782 (local address 127.8.154.60:39223)
0809 08:52:32.435815 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:32.435818 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:32.435835 (+ 17us) server_negotiation.cc:407] Connection header received
0809 08:52:32.435853 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:32.435854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:32.435870 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:32.435951 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:32.436674 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.437466 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.438497 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.438987 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.439069 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.442079 (+ 3010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:32.442086 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:32.442087 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:32.442094 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:32.444242 (+ 2148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.444610 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.444612 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.444613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.444666 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.444905 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.444907 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.444907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.445063 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:32.445200 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:32.445289 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 08:52:32.445405 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":95,"threads_started":1}
Aug 09 08:52:32 dist-test-slave-t3q3 krb5kdc[18975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265552, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.60@KRBTEST.COM for oryx/127.8.154.61@KRBTEST.COM
I20260809 08:52:32.450500 20234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.443968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.61:41867 (local address 127.0.0.1:41112)
0809 08:52:32.444121 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:32.444134 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:32.444185 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:32.444348 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:32.444349 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:32.444463 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:32.444492 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.444494 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.445352 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.445354 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.446112 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.446114 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.446519 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.446520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.446582 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.446927 (+ 345us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:32.446932 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:32.448037 (+ 1105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:32.449439 (+ 1402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:32.449440 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:32.449440 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:32.449616 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:32.449841 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:32.449842 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:32.449842 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:32.449907 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:32.450182 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:32.450183 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:32.450224 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 08:52:32.450311 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 08:52:32.450408 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 08:52:32.450512 20093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.444086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41112 (local address 127.8.154.61:41867)
0809 08:52:32.444127 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:32.444130 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:32.444170 (+ 40us) server_negotiation.cc:407] Connection header received
0809 08:52:32.444257 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:32.444258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:32.444268 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:32.444325 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:32.444577 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.445263 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.446222 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.446431 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.446509 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.448177 (+ 1668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:32.448178 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:32.448178 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:32.448193 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:32.449337 (+ 1144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.449704 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.449705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.449706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.449768 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.449984 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.449985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.449985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.450104 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:32.450176 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:32.450339 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 08:52:32.450425 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 08:52:32.450992 20230 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.154.60" port: 39223 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 08:52:32 dist-test-slave-t3q3 krb5kdc[18975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265552, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.60@KRBTEST.COM for oryx/127.8.154.60@KRBTEST.COM
I20260809 08:52:32.457401 20234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.451271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:39223 (local address 127.0.0.1:33784)
0809 08:52:32.451306 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:32.451315 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:32.451361 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:32.451474 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:32.451475 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:32.451604 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:32.451628 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.451629 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.452383 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.452384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.452841 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.452842 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.453341 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.453343 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.453391 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.453702 (+ 311us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:32.453705 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:32.454814 (+ 1109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:32.456581 (+ 1767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:32.456582 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:32.456582 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:32.456750 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:32.456927 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:32.456927 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:32.456928 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:32.456980 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:32.457165 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:32.457165 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:32.457199 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 08:52:32.457248 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 08:52:32.457322 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260809 08:52:32.457494 20235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.451305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33784 (local address 127.8.154.60:39223)
0809 08:52:32.451337 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:32.451339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:32.451348 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:52:32.451407 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:32.451408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:32.451417 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:32.451470 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:32.451700 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.452294 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.452985 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.453262 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.453344 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.454927 (+ 1583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:32.454928 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:32.454929 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:32.454937 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:32.456489 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.456817 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.456818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.456818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.456867 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.457027 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.457028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.457028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.457115 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:32.457160 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:32.457266 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 08:52:32.457367 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":18}
I20260809 08:52:32.458830 20230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c: Bootstrap starting.
I20260809 08:52:32.459749 20230 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:32.460146 20230 log.cc:824] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:32.460903 20230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c: No bootstrap required, opened a new log
I20260809 08:52:32.462364 20230 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 41867 } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 39223 } }
I20260809 08:52:32.462512 20230 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:32.462560 20230 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88b956d3e2a24162ac5e1709d2832b3c, State: Initialized, Role: FOLLOWER
I20260809 08:52:32.462697 20230 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 41867 } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 39223 } }
I20260809 08:52:32.463001 20237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 41867 } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 39223 } } }
I20260809 08:52:32.463056 20230 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:52:32.463109 20237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [sys.catalog]: This master's current role is: FOLLOWER
W20260809 08:52:32.464862 20253 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:52:32.465020 20253 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:52:32.467238 20122 master_runner.cc:428] Detected that this master 88b956d3e2a24162ac5e1709d2832b3c is joining an existing cluster
I20260809 08:52:32.467308 20122 master_runner.cc:432] Initiating AddMaster RPC to add 127.8.154.60:39223
I20260809 08:52:32.476265 20267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.468410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:39223 (local address 127.0.0.1:33790)
0809 08:52:32.468938 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:32.468969 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:32.469045 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:32.469207 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:32.469208 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:32.469585 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:32.469677 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.469679 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.470579 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.470580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.471466 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.471468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.472016 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.472017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.472103 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.472674 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:32.472678 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:32.473045 (+ 367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:32.474963 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:32.474964 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:32.474964 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:32.475183 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:32.475470 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:32.475471 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:32.475471 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:32.475539 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:32.475834 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:32.475835 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:32.475887 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 08:52:32.475946 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 08:52:32.476052 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"mutex_wait_us":1,"thread_start_us":91,"threads_started":1}
I20260809 08:52:32.476366 20235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.468423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33790 (local address 127.8.154.60:39223)
0809 08:52:32.468466 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:32.468469 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:32.469022 (+ 553us) server_negotiation.cc:407] Connection header received
0809 08:52:32.469114 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:32.469115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:32.469124 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:32.469151 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:32.469765 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.470482 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.471570 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.471840 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.471878 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.473149 (+ 1271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:32.473150 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:32.473151 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:32.473158 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:32.474873 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.475311 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.475312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.475312 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.475376 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.475623 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.475624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.475625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.475759 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:32.475835 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:32.476008 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 08:52:32.476135 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 08:52:32.496743 20093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.469013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41126 (local address 127.8.154.61:41867)
0809 08:52:32.469052 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:32.469054 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:32.469225 (+ 171us) server_negotiation.cc:407] Connection header received
0809 08:52:32.469324 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:32.469325 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:32.469335 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:32.469364 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:32.469823 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.470604 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.471686 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.471999 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.472044 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.493513 (+ 21469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:32.493515 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:32.493515 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:32.493523 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:32.495095 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.495488 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.495490 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.495490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.495556 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.495968 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.495969 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.495970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.496155 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:32.496236 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:32.496523 (+ 287us) server_negotiation.cc:299] Negotiation successful
0809 08:52:32.496626 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 08:52:32.496841 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 19157
I20260809 08:52:32.496858 19126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.469036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54804 (local address 127.8.154.62:45607)
0809 08:52:32.469096 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:32.469098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:32.469105 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:52:32.469126 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:32.469127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:32.469137 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:32.469187 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:32.469774 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.470530 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.471580 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.471898 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.471938 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.493484 (+ 21546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:32.493486 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:32.493487 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:32.493496 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:32.495389 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.495804 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.495806 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.495806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.495870 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.496118 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.496119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.496119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.496296 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:32.496366 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:32.496671 (+ 305us) server_negotiation.cc:299] Negotiation successful
0809 08:52:32.496756 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 08:52:32.497279 20266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.468394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45607 (local address 127.0.0.1:54804)
0809 08:52:32.468612 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:32.468621 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:32.468666 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:32.469231 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:32.469232 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:32.469610 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:32.469677 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.469679 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.470631 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.470632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.471492 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.471494 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.472056 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.472057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.472136 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.492721 (+ 20585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:32.492729 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:32.493359 (+ 630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:32.495514 (+ 2155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:32.495515 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:32.495515 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:32.495725 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:32.495942 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:32.495943 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:32.495943 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:32.496012 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:32.496501 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:32.496502 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:32.496550 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 08:52:32.497032 (+ 482us) client_negotiation.cc:240] Negotiation successful
0809 08:52:32.497152 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":77,"threads_started":1}
I20260809 08:52:32.499742 20265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.468621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.61:41867 (local address 127.0.0.1:41126)
0809 08:52:32.469153 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:32.469163 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:32.469251 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:32.469429 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:32.469429 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:32.469657 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:32.469726 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.469728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.470729 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.470730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.471582 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.471586 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.472182 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.472182 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.472263 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.492948 (+ 20685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:32.492953 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:32.493354 (+ 401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:32.495214 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:32.495215 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:32.495215 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:32.495404 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:32.495640 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:32.495642 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:32.495642 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:32.495717 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:32.496228 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:32.496229 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:32.496393 (+ 164us) client_negotiation.cc:769] Sending connection context
0809 08:52:32.496884 (+ 491us) client_negotiation.cc:240] Negotiation successful
0809 08:52:32.499562 (+ 2678us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"mutex_wait_us":2,"thread_start_us":255,"threads_started":1}
I20260809 08:52:32.524900 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.1:44029
--local_ip_for_outbound_sockets=127.8.154.1
--tserver_master_addrs=127.8.154.62:45607
--webserver_port=35045
--webserver_interface=127.8.154.1
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.1.keytab
--principal=oryx/127.8.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:45337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.154.62:45607,127.8.154.61:41867
--tserver_master_addrs=127.8.154.62:45607,127.8.154.61:41867,127.8.154.60:39223 with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 08:52:32 dist-test-slave-t3q3 krb5kdc[18975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265551, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.62@KRBTEST.COM for oryx/127.8.154.60@KRBTEST.COM
I20260809 08:52:32.530310 20235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.522004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33802 (local address 127.8.154.60:39223)
0809 08:52:32.522048 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:32.522050 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:32.522061 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:52:32.522133 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:32.522134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:32.522145 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:32.522221 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:32.522641 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.523416 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.524683 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.524949 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.525026 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.527204 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:32.527206 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:32.527206 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:32.527221 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:32.528972 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.529402 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.529403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.529404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.529468 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.529717 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.529718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.529718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.529859 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:32.529931 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:32.530120 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 08:52:32.530204 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 08:52:32.530357 19971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.521939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:39223 (local address 127.0.0.1:33802)
0809 08:52:32.522003 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:32.522015 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:32.522059 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:32.522237 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:32.522238 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:32.522462 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:32.522527 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.522530 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.523516 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.523518 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.524565 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.524569 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.525023 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.525024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.525095 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.525560 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:32.525566 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:32.527104 (+ 1538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:32.529062 (+ 1958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:32.529063 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:32.529064 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:32.529319 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:32.529559 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:32.529560 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:32.529560 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:32.529639 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:32.529939 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:32.529940 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:32.530000 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 08:52:32.530082 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 08:52:32.530204 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 08:52:32.531219 19049 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.8.154.60:39223: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c"
member_type: NON_VOTER
last_known_addr {
host: "127.8.154.60"
port: 39223
}
attrs {
promote: true
}
}
dest_uuid: "a22f631ee17d42f2a60289505b9f1fe4"
cas_config_opid_index: 5
I20260809 08:52:32.531473 19049 consensus_queue.cc:237] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: NON_VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: true } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: NON_VOTER last_known_addr { host: "127.8.154.60" port: 39223 } attrs { promote: true } }
I20260809 08:52:32.531848 19974 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 88b956d3e2a24162ac5e1709d2832b3c (127.8.154.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: NON_VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: true } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: NON_VOTER last_known_addr { host: "127.8.154.60" port: 39223 } attrs { promote: true } } }
I20260809 08:52:32.532117 20186 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:32.533051 19974 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.8.154.60:39223
I20260809 08:52:32.533125 19974 sys_catalog.cc:455] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a22f631ee17d42f2a60289505b9f1fe4" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: NON_VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: true } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: NON_VOTER last_known_addr { host: "127.8.154.60" port: 39223 } attrs { promote: true } } }
I20260809 08:52:32.533210 19974 sys_catalog.cc:458] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:32.533227 20186 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [term 1 FOLLOWER]: Refusing update from remote peer a22f631ee17d42f2a60289505b9f1fe4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 08:52:32.533650 19972 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: NON_VOTER last_known_addr { host: "127.8.154.60" port: 39223 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:52:32.534116 20122 master.cc:537] Master@127.8.154.60:39223 shutting down...
I20260809 08:52:32.535040 20186 raft_consensus.cc:858] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 41867 } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 39223 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: NON_VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: NON_VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: true } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: NON_VOTER last_known_addr { host: "127.8.154.60" port: 39223 } attrs { promote: true } }
W20260809 08:52:32.535115 20186 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 41867 } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 39223 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: NON_VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: NON_VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: true } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: NON_VOTER last_known_addr { host: "127.8.154.60" port: 39223 } attrs { promote: true } }
I20260809 08:52:32.535609 20186 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER a9f86b88da034484ab3f19ad3b945ade (127.8.154.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: NON_VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: true } } }
I20260809 08:52:32.537345 20237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a22f631ee17d42f2a60289505b9f1fe4. Latest consensus state: current_term: 1 leader_uuid: "a22f631ee17d42f2a60289505b9f1fe4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: NON_VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: true } } }
I20260809 08:52:32.537424 20237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260809 08:52:32.537461 20237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a22f631ee17d42f2a60289505b9f1fe4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: NON_VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: true } } }
I20260809 08:52:32.537498 20237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260809 08:52:32.537525 20237 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a22f631ee17d42f2a60289505b9f1fe4" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: NON_VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: true } } }
I20260809 08:52:32.537576 20237 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260809 08:52:32.538560 19972 sys_catalog.cc:455] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a22f631ee17d42f2a60289505b9f1fe4" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: NON_VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: true } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: NON_VOTER last_known_addr { host: "127.8.154.60" port: 39223 } attrs { promote: true } } }
I20260809 08:52:32.538666 19972 sys_catalog.cc:458] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:32.539134 20273 mvcc.cc:205] Tried to move back new op lower bound from 7316543698695147520 to 7316543698561781760. Current Snapshot: MvccSnapshot[applied={T|T < 7316543698571022336}]
I20260809 08:52:32.539346 20122 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260809 08:52:32.539422 20122 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260809 08:52:32.539450 20122 tablet_replica.cc:330] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c: stopping tablet replica
W20260809 08:52:32.539500 19031 consensus_peers.cc:487] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 -> Peer 88b956d3e2a24162ac5e1709d2832b3c (127.8.154.60:39223): Couldn't send request to peer 88b956d3e2a24162ac5e1709d2832b3c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 08:52:32.539929 19971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.533492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.61:41867 (local address 127.0.0.1:41128)
0809 08:52:32.534462 (+ 970us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:32.534474 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:32.534526 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:32.534763 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:32.534764 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:32.534765 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:32.534800 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.534802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.535770 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.535771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.537371 (+ 1600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.537375 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.538406 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.538407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.539687 (+ 1280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.539714 (+ 27us) client_negotiation.cc:769] Sending connection context
0809 08:52:32.539788 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 08:52:32.539835 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":952}
I20260809 08:52:32.539944 20093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.533989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41128 (local address 127.8.154.61:41867)
0809 08:52:32.534425 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:32.534428 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:32.534591 (+ 163us) server_negotiation.cc:407] Connection header received
0809 08:52:32.534618 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:32.534619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:32.534646 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:32.534703 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:32.534877 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.535666 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.537484 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.538194 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.538277 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.538365 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:32.539811 (+ 1446us) server_negotiation.cc:299] Negotiation successful
0809 08:52:32.539858 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405}
I20260809 08:52:32.540524 20051 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [term 1 LEARNER]: Refusing update from remote peer a22f631ee17d42f2a60289505b9f1fe4: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 08:52:32.540759 19974 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: NON_VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 08:52:32.541327 20051 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 88b956d3e2a24162ac5e1709d2832b3c (127.8.154.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: NON_VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: true } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: NON_VOTER last_known_addr { host: "127.8.154.60" port: 39223 } attrs { promote: true } } }
I20260809 08:52:32.542591 19974 raft_consensus.cc:1050] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4: attempting to promote NON_VOTER a9f86b88da034484ab3f19ad3b945ade to VOTER
I20260809 08:52:32.542651 20096 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [sys.catalog]: SysCatalogTable state changed. Reason: New leader a22f631ee17d42f2a60289505b9f1fe4. Latest consensus state: current_term: 1 leader_uuid: "a22f631ee17d42f2a60289505b9f1fe4" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: NON_VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: true } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: NON_VOTER last_known_addr { host: "127.8.154.60" port: 39223 } attrs { promote: true } } }
I20260809 08:52:32.542732 20096 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [sys.catalog]: This master's current role is: LEARNER
I20260809 08:52:32.542769 20096 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a22f631ee17d42f2a60289505b9f1fe4" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: NON_VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: true } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: NON_VOTER last_known_addr { host: "127.8.154.60" port: 39223 } attrs { promote: true } } }
I20260809 08:52:32.542809 20096 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [sys.catalog]: This master's current role is: LEARNER
I20260809 08:52:32.542760 19974 consensus_queue.cc:237] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: false } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: NON_VOTER last_known_addr { host: "127.8.154.60" port: 39223 } attrs { promote: true } }
I20260809 08:52:32.543232 20051 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [term 1 LEARNER]: Refusing update from remote peer a22f631ee17d42f2a60289505b9f1fe4: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 08:52:32.543419 19972 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260809 08:52:32.543689 19031 consensus_peers.cc:487] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 -> Peer 88b956d3e2a24162ac5e1709d2832b3c (127.8.154.60:39223): Couldn't send request to peer 88b956d3e2a24162ac5e1709d2832b3c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 08:52:32.544340 19974 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, a9f86b88da034484ab3f19ad3b945ade (127.8.154.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: false } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: NON_VOTER last_known_addr { host: "127.8.154.60" port: 39223 } attrs { promote: true } } }
I20260809 08:52:32.544512 20051 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, a9f86b88da034484ab3f19ad3b945ade (127.8.154.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: false } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: NON_VOTER last_known_addr { host: "127.8.154.60" port: 39223 } attrs { promote: true } } }
I20260809 08:52:32.545186 19974 sys_catalog.cc:455] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a22f631ee17d42f2a60289505b9f1fe4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: false } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: NON_VOTER last_known_addr { host: "127.8.154.60" port: 39223 } attrs { promote: true } } }
I20260809 08:52:32.545264 19974 sys_catalog.cc:458] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:32.545312 19974 sys_catalog.cc:455] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a22f631ee17d42f2a60289505b9f1fe4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: false } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: NON_VOTER last_known_addr { host: "127.8.154.60" port: 39223 } attrs { promote: true } } }
I20260809 08:52:32.545351 19974 sys_catalog.cc:458] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:32.545581 20096 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a22f631ee17d42f2a60289505b9f1fe4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: false } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: NON_VOTER last_known_addr { host: "127.8.154.60" port: 39223 } attrs { promote: true } } }
I20260809 08:52:32.545671 20096 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:52:32.563030 20122 master.cc:552] Master@127.8.154.60:39223 shutdown complete.
I20260809 08:52:32.563095 20122 master_runner.cc:305] Clearing existing system tablet
I20260809 08:52:32.563242 20122 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:32.563319 20122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:32.564147 20122 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260809 08:52:32.564767 20285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:32.564934 20122 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:32.564983 20122 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:32.565006 20122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-2/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-2/wal
uuid: "88b956d3e2a24162ac5e1709d2832b3c"
format_stamp: "Formatted at 2026-08-09 08:52:32 on dist-test-slave-t3q3"
server_key: "18e61d593f0c346db983c3e1f796b7d3"
server_key_iv: "def61ff707c4346a01d5ea06fc4a9131"
server_key_version: "encryptionkey@0"
I20260809 08:52:32.565714 20122 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:52:32.567268 20122 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 08:52:32.567340 20122 log.cc:1196] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 08:52:32.567678 20122 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c: Deleting consensus metadata
I20260809 08:52:32.567809 20122 master_runner.cc:315] Copying system tablet from 127.8.154.62:45607
I20260809 08:52:32.568481 20122 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c: tablet copy: Beginning tablet copy session from remote peer at address 127.8.154.62:45607
I20260809 08:52:32.576392 19126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.568976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54806 (local address 127.8.154.62:45607)
0809 08:52:32.569023 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:32.569025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:32.569064 (+ 39us) server_negotiation.cc:407] Connection header received
0809 08:52:32.569092 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:32.569093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:32.569103 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:32.569163 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:32.569553 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.570381 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.571446 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.571767 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.571851 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.573019 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:32.573020 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:32.573021 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:32.573028 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:32.574968 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.575397 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.575398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.575399 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.575469 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.575713 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.575714 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.575715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.575852 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:32.575902 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:32.576200 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 08:52:32.576297 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 08:52:32.576997 20294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.568791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45607 (local address 127.0.0.1:54806)
0809 08:52:32.568941 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:32.568955 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:32.569021 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:32.569180 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:32.569181 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:32.569388 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:32.569446 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.569448 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.570476 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.570477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.571343 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.571345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.571860 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.571861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.571921 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.572480 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:32.572486 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:32.572899 (+ 413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:32.575059 (+ 2160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:32.575061 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:32.575061 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:32.575304 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:32.575562 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:32.575563 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:32.575563 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:32.575634 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:32.575958 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:32.575959 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:32.576010 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 08:52:32.576766 (+ 756us) client_negotiation.cc:240] Negotiation successful
0809 08:52:32.576888 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":79,"threads_started":1}
I20260809 08:52:32.577596 19089 tablet_copy_service.cc:140] P a22f631ee17d42f2a60289505b9f1fe4: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 88b956d3e2a24162ac5e1709d2832b3c ({username='oryx', principal='oryx/127.8.154.60@KRBTEST.COM'} at 127.0.0.1:54806)
I20260809 08:52:32.577669 19089 tablet_copy_service.cc:161] P a22f631ee17d42f2a60289505b9f1fe4: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 88b956d3e2a24162ac5e1709d2832b3c at {username='oryx', principal='oryx/127.8.154.60@KRBTEST.COM'} at 127.0.0.1:54806: session id = 88b956d3e2a24162ac5e1709d2832b3c-00000000000000000000000000000000
I20260809 08:52:32.578248 19089 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 08:52:32.578758 20122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 08:52:32.580003 20122 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c: tablet copy: Starting download of 0 data blocks...
I20260809 08:52:32.580240 20122 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c: tablet copy: Starting download of 1 WAL segments...
I20260809 08:52:32.581032 20122 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 08:52:32.582003 19089 tablet_copy_service.cc:342] P a22f631ee17d42f2a60289505b9f1fe4: Request end of tablet copy session 88b956d3e2a24162ac5e1709d2832b3c-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.8.154.60@KRBTEST.COM'} at 127.0.0.1:54806
I20260809 08:52:32.582060 19089 tablet_copy_service.cc:433] P a22f631ee17d42f2a60289505b9f1fe4: ending tablet copy session 88b956d3e2a24162ac5e1709d2832b3c-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 88b956d3e2a24162ac5e1709d2832b3c
I20260809 08:52:32.585264 20122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:32.586445 20295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:32.589195 20298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:32.589442 20122 server_base.cc:1034] running on GCE node
W20260809 08:52:32.589594 20296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:32.589772 20122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:32.589912 20122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:32.591038 20122 hybrid_clock.cc:648] HybridClock initialized: now 1786265552591027 us; error 31 us; skew 500 ppm
Aug 09 08:52:32 dist-test-slave-t3q3 krb5kdc[18975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265552, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:32.592679 20122 init.cc:377] Logged in from keytab as oryx/127.8.154.60@KRBTEST.COM (short username oryx)
I20260809 08:52:32.592976 20122 webserver.cc:460] Webserver started at http://127.8.154.60:33429/ using document root <none> and password file <none>
I20260809 08:52:32.593112 20122 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:32.593149 20122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:32.593947 20122 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:32.594554 20308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:32.594730 20122 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:32.594787 20122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-2/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-2/wal
uuid: "88b956d3e2a24162ac5e1709d2832b3c"
format_stamp: "Formatted at 2026-08-09 08:52:32 on dist-test-slave-t3q3"
server_key: "18e61d593f0c346db983c3e1f796b7d3"
server_key_iv: "def61ff707c4346a01d5ea06fc4a9131"
server_key_version: "encryptionkey@0"
I20260809 08:52:32.594848 20122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:32.608795 20122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:32.610991 20122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:32.614982 20122 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.60:39223
I20260809 08:52:32.615024 20387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.60:39223 every 8 connection(s)
I20260809 08:52:32.615407 20122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/master-2/data/info.pb
I20260809 08:52:32.616926 20390 sys_catalog.cc:263] Verifying existing consensus state
I20260809 08:52:32.617496 20390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c: Bootstrap starting.
I20260809 08:52:32.619982 20390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 08:52:32.620257 20390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c: Bootstrap complete.
I20260809 08:52:32.620494 20390 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: false } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: NON_VOTER last_known_addr { host: "127.8.154.60" port: 39223 } attrs { promote: true } }
I20260809 08:52:32.620592 20390 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 88b956d3e2a24162ac5e1709d2832b3c, State: Initialized, Role: LEARNER
I20260809 08:52:32.620713 20390 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: false } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: NON_VOTER last_known_addr { host: "127.8.154.60" port: 39223 } attrs { promote: true } }
I20260809 08:52:32.620908 20390 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:52:32.622195 20394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: false } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: NON_VOTER last_known_addr { host: "127.8.154.60" port: 39223 } attrs { promote: true } } }
I20260809 08:52:32.622567 20394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [sys.catalog]: This master's current role is: LEARNER
I20260809 08:52:32.623287 20410 catalog_manager.cc:1259] Loaded cluster ID: ac149d96c0c84228a4d9a3175bba4c67
I20260809 08:52:32.623342 20410 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c: loading cluster ID for follower catalog manager: success
I20260809 08:52:32.624465 20410 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c: acquiring CA information for follower catalog manager: success
I20260809 08:52:32.624792 20410 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 08:52:32.633687 20271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:32.633872 20271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:32.633895 20271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:32.635036 20271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:32.635078 20271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:32.635104 20271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.1
I20260809 08:52:32.636672 20271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:45337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.1.keytab
--principal=oryx/127.8.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.1:44029
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.154.1
--webserver_port=35045
--tserver_master_addrs=127.8.154.62:45607,127.8.154.61:41867,127.8.154.60:39223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.1
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:32.636943 20271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:32.637171 20271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:32.642264 20419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:32.642287 20417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:32.642284 20416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:32.642465 20271 server_base.cc:1034] running on GCE node
I20260809 08:52:32.642748 20271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:32.643008 20271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:32.644124 20271 hybrid_clock.cc:648] HybridClock initialized: now 1786265552644107 us; error 26 us; skew 500 ppm
Aug 09 08:52:32 dist-test-slave-t3q3 krb5kdc[18975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265552, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:32.646608 20271 init.cc:377] Logged in from keytab as oryx/127.8.154.1@KRBTEST.COM (short username oryx)
I20260809 08:52:32.647028 20271 webserver.cc:460] Webserver started at http://127.8.154.1:35045/ using document root <none> and password file <none>
I20260809 08:52:32.647264 20271 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:32.647349 20271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:32.648821 20271 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:52:32.649418 20430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:32.649587 20271 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:52:32.649694 20271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-0/wal
uuid: "ad6563f6630440b2afc6ba67e966bebf"
format_stamp: "Formatted at 2026-08-09 08:52:31 on dist-test-slave-t3q3"
server_key: "bc24f14331862f643254b0b14d7f4b8a"
server_key_iv: "bb502c2758e57f290bb659154d1bdefb"
server_key_version: "encryptionkey@0"
I20260809 08:52:32.650038 20271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:32.659564 20271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:32.661530 20271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:32.661693 20271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:32.662072 20271 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:32.662395 20271 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:32.662451 20271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:32.662492 20271 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:32.662516 20271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:32.670343 20271 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.1:44029
I20260809 08:52:32.670408 20612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.1:44029 every 8 connection(s)
I20260809 08:52:32.670786 20271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
I20260809 08:52:32.672003 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 20271
I20260809 08:52:32.672195 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 19399
Aug 09 08:52:32 dist-test-slave-t3q3 krb5kdc[18975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265552, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.60@KRBTEST.COM
Aug 09 08:52:32 dist-test-slave-t3q3 krb5kdc[18975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265552, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.61@KRBTEST.COM
Aug 09 08:52:32 dist-test-slave-t3q3 krb5kdc[18975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265552, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM
I20260809 08:52:32.690662 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.2:32787
--local_ip_for_outbound_sockets=127.8.154.2
--tserver_master_addrs=127.8.154.62:45607
--webserver_port=36863
--webserver_interface=127.8.154.2
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.2.keytab
--principal=oryx/127.8.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:45337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.154.62:45607,127.8.154.61:41867
--tserver_master_addrs=127.8.154.62:45607,127.8.154.61:41867,127.8.154.60:39223 with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:52:32.690940 20624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.672746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:43635 (local address 127.8.154.60:39223)
0809 08:52:32.680130 (+ 7384us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:32.680132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:32.680141 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:52:32.680166 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:32.680167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:32.680186 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:32.680269 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:32.680628 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.681675 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.682932 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.683228 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.683279 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.687202 (+ 3923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:32.687203 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:32.687203 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:32.687211 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:32.689128 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.689730 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.689731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.689731 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.689796 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.690083 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.690084 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.690084 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.690242 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:32.690305 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:32.690734 (+ 429us) server_negotiation.cc:299] Negotiation successful
0809 08:52:32.690846 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7349,"thread_start_us":1898,"threads_started":1}
I20260809 08:52:32.690948 19126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.671631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:40241 (local address 127.8.154.62:45607)
0809 08:52:32.671680 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:32.671682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:32.671953 (+ 271us) server_negotiation.cc:407] Connection header received
0809 08:52:32.676251 (+ 4298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:32.676253 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:32.676269 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:32.676302 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:32.677491 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.678289 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.680700 (+ 2411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.681107 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.681177 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.687352 (+ 6175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:32.687354 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:32.687354 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:32.687363 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:32.689217 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.689760 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.689762 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.689762 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.689835 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.690106 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.690107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.690107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.690258 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:32.690310 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:32.690686 (+ 376us) server_negotiation.cc:299] Negotiation successful
0809 08:52:32.690775 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 08:52:32.691428 20093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.672110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:36413 (local address 127.8.154.61:41867)
0809 08:52:32.672159 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:32.672161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:32.672227 (+ 66us) server_negotiation.cc:407] Connection header received
0809 08:52:32.676391 (+ 4164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:32.676393 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:32.676405 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:32.676434 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:32.677491 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.678280 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.680095 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.680512 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.680624 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.686860 (+ 6236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:32.686861 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:32.686862 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:32.686871 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:32.689638 (+ 2767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.690022 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.690023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.690023 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.690094 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.690291 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.690291 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.690292 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.690427 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:32.690487 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:32.691303 (+ 816us) server_negotiation.cc:299] Negotiation successful
0809 08:52:32.691369 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 08:52:32.691957 20621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.671602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:39223 (local address 127.8.154.1:43635)
0809 08:52:32.671810 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:32.671862 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:32.672604 (+ 742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:32.680291 (+ 7687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:32.680292 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:32.680483 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:32.680547 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.680549 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.681795 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.681797 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.682837 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.682840 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.683325 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.683326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.683394 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.683936 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:32.683949 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:32.687099 (+ 3150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:32.689227 (+ 2128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:32.689235 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:32.689237 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:32.689642 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:32.689869 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:32.689870 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:32.689870 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:32.690009 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:32.690332 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:32.690339 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:32.690508 (+ 169us) client_negotiation.cc:769] Sending connection context
0809 08:52:32.690639 (+ 131us) client_negotiation.cc:240] Negotiation successful
0809 08:52:32.691702 (+ 1063us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":863,"spinlock_wait_cycles":7040,"thread_start_us":112,"threads_started":1}
I20260809 08:52:32.691957 20622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.671921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.61:41867 (local address 127.8.154.1:36413)
0809 08:52:32.672167 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:32.672178 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:32.676287 (+ 4109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:32.676494 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:32.676494 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:32.677051 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:32.677369 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.677375 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.678432 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.678433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.679955 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.679958 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.681244 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.681245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.681404 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.685067 (+ 3663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:32.685073 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:32.686741 (+ 1668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:32.689728 (+ 2987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:32.689729 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:32.689729 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:32.689943 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:32.690169 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:32.690169 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:32.690170 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:32.690222 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:32.690513 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:32.690514 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:32.690567 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 08:52:32.690639 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 08:52:32.691749 (+ 1110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"mutex_wait_us":175,"thread_start_us":59,"threads_started":1}
I20260809 08:52:32.692428 20619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.671602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45607 (local address 127.8.154.1:40241)
0809 08:52:32.671811 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:32.671862 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:32.676142 (+ 4280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:32.676468 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:32.676473 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:32.677052 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:32.677369 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.677375 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.678433 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.678433 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.679955 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.679958 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.681646 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.681647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.681708 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.685547 (+ 3839us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:32.685551 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:32.687242 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:32.689313 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:32.689314 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:32.689314 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:32.689661 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:32.689906 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:32.689907 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:32.689907 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:32.690028 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:32.690346 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:32.690347 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:32.690507 (+ 160us) client_negotiation.cc:769] Sending connection context
0809 08:52:32.691081 (+ 574us) client_negotiation.cc:240] Negotiation successful
0809 08:52:32.692329 (+ 1248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":1197,"thread_start_us":96,"threads_started":1}
I20260809 08:52:32.692745 20617 heartbeater.cc:347] Connected to a master server at 127.8.154.61:41867
I20260809 08:52:32.692751 20615 heartbeater.cc:347] Connected to a master server at 127.8.154.62:45607
I20260809 08:52:32.692886 20615 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:32.692885 20617 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:32.693121 20614 heartbeater.cc:347] Connected to a master server at 127.8.154.60:39223
I20260809 08:52:32.693106 20615 heartbeater.cc:511] Master 127.8.154.62:45607 requested a full tablet report, sending...
I20260809 08:52:32.693111 20617 heartbeater.cc:511] Master 127.8.154.61:41867 requested a full tablet report, sending...
I20260809 08:52:32.693164 20614 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:32.693248 20614 heartbeater.cc:511] Master 127.8.154.60:39223 requested a full tablet report, sending...
I20260809 08:52:32.693569 19049 ts_manager.cc:194] Re-registered known tserver with Master: ad6563f6630440b2afc6ba67e966bebf (127.8.154.1:44029)
I20260809 08:52:32.693713 20035 ts_manager.cc:194] Registered new tserver with Master: ad6563f6630440b2afc6ba67e966bebf (127.8.154.1:44029)
I20260809 08:52:32.694116 19049 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.154.1@KRBTEST.COM'} at 127.8.154.1:40241
I20260809 08:52:32.694496 20325 ts_manager.cc:194] Registered new tserver with Master: ad6563f6630440b2afc6ba67e966bebf (127.8.154.1:44029)
W20260809 08:52:32.792807 20625 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:32.793020 20625 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:32.793045 20625 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:32.794126 20625 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:32.794157 20625 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:32.794183 20625 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.2
I20260809 08:52:32.795653 20625 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:45337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.2.keytab
--principal=oryx/127.8.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.2:32787
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.154.2
--webserver_port=36863
--tserver_master_addrs=127.8.154.62:45607,127.8.154.61:41867,127.8.154.60:39223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20625
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.2
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:32.795878 20625 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:32.796149 20625 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:32.801082 20630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:32.801082 20634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:32.801090 20632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:32.801350 20625 server_base.cc:1034] running on GCE node
I20260809 08:52:32.801571 20625 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:32.801856 20625 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:32.803018 20625 hybrid_clock.cc:648] HybridClock initialized: now 1786265552803010 us; error 33 us; skew 500 ppm
Aug 09 08:52:32 dist-test-slave-t3q3 krb5kdc[18975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265552, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:32.806226 20625 init.cc:377] Logged in from keytab as oryx/127.8.154.2@KRBTEST.COM (short username oryx)
I20260809 08:52:32.806780 20625 webserver.cc:460] Webserver started at http://127.8.154.2:36863/ using document root <none> and password file <none>
I20260809 08:52:32.806988 20625 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:32.807049 20625 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:32.808616 20625 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 08:52:32.809229 20645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:32.809397 20625 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 08:52:32.809494 20625 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-1/wal
uuid: "a7984b066bfa4a7cbea4fd04bd72c6d8"
format_stamp: "Formatted at 2026-08-09 08:52:31 on dist-test-slave-t3q3"
server_key: "7f21a1668823ca6658918f5182638add"
server_key_iv: "2484ec9112f6100b098ab68791cec17b"
server_key_version: "encryptionkey@0"
I20260809 08:52:32.809814 20625 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:32.830022 20625 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:32.832440 20625 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:32.832558 20625 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:32.832760 20625 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:32.833050 20625 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:32.833081 20625 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:32.833103 20625 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:32.833115 20625 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:32.843142 20625 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.2:32787
I20260809 08:52:32.843227 20828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.2:32787 every 8 connection(s)
I20260809 08:52:32.843637 20625 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
I20260809 08:52:32.847261 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 20625
I20260809 08:52:32.847450 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 19606
Aug 09 08:52:32 dist-test-slave-t3q3 krb5kdc[18975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265552, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.61@KRBTEST.COM
Aug 09 08:52:32 dist-test-slave-t3q3 krb5kdc[18975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265552, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.60@KRBTEST.COM
I20260809 08:52:32.880465 20624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.844914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:37481 (local address 127.8.154.60:39223)
0809 08:52:32.844957 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:32.844959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:32.844971 (+ 12us) server_negotiation.cc:407] Connection header received
0809 08:52:32.845542 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:32.845544 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:32.845560 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:32.845615 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:32.846346 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.846863 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.865951 (+ 19088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.866290 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.866365 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.874014 (+ 7649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:32.874016 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:32.874016 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:32.874027 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:32.878425 (+ 4398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.879072 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.879074 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.879074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.879146 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.879505 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.879507 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.879507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.879671 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:32.879725 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:32.880199 (+ 474us) server_negotiation.cc:299] Negotiation successful
0809 08:52:32.880316 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Aug 09 08:52:32 dist-test-slave-t3q3 krb5kdc[18975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265552, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM
I20260809 08:52:32.880723 20093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.845085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:48243 (local address 127.8.154.61:41867)
0809 08:52:32.845150 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:32.845152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:32.845244 (+ 92us) server_negotiation.cc:407] Connection header received
0809 08:52:32.845552 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:32.845553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:32.845566 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:32.845599 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:32.846361 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.847142 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.866768 (+ 19626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.867105 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.867148 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.873163 (+ 6015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:32.873166 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:32.873166 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:32.873177 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:32.879274 (+ 6097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.879738 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.879740 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.879740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.879808 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.880100 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.880101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.880102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.880248 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:32.880299 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:32.880538 (+ 239us) server_negotiation.cc:299] Negotiation successful
0809 08:52:32.880624 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 08:52:32.881188 20837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.844563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:39223 (local address 127.8.154.2:37481)
0809 08:52:32.844758 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:32.844789 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:32.845428 (+ 639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:32.845702 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:32.845705 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:32.846038 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:32.846242 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.846246 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.846979 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.846985 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.865800 (+ 18815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.865807 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.866450 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.866452 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.866615 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.871212 (+ 4597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:32.871217 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:32.873867 (+ 2650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:32.878546 (+ 4679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:32.878557 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:32.878561 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:32.878911 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:32.879233 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:32.879234 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:32.879234 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:32.879408 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:32.879784 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:32.879792 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:32.879992 (+ 200us) client_negotiation.cc:769] Sending connection context
0809 08:52:32.880635 (+ 643us) client_negotiation.cc:240] Negotiation successful
0809 08:52:32.880857 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":121,"threads_started":1}
I20260809 08:52:32.881191 20838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.844871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.61:41867 (local address 127.8.154.2:48243)
0809 08:52:32.845161 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:32.845170 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:32.845447 (+ 277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:32.845703 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:32.845705 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:32.846069 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:32.846244 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.846249 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.847244 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.847244 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.866646 (+ 19402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.866648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.867226 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.867227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.867291 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.869489 (+ 2198us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:32.869500 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:32.871409 (+ 1909us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:32.879373 (+ 7964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:32.879373 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:32.879374 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:32.879634 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:32.879909 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:32.879910 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:32.879910 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:32.879978 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:32.880358 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:32.880359 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:32.880422 (+ 63us) client_negotiation.cc:769] Sending connection context
0809 08:52:32.880844 (+ 422us) client_negotiation.cc:240] Negotiation successful
0809 08:52:32.880971 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":105,"thread_start_us":55,"threads_started":1}
I20260809 08:52:32.881759 20829 heartbeater.cc:347] Connected to a master server at 127.8.154.60:39223
I20260809 08:52:32.881870 20829 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:32.882086 20829 heartbeater.cc:511] Master 127.8.154.60:39223 requested a full tablet report, sending...
I20260809 08:52:32.882237 20834 heartbeater.cc:347] Connected to a master server at 127.8.154.61:41867
I20260809 08:52:32.882294 20834 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:32.882383 20834 heartbeater.cc:511] Master 127.8.154.61:41867 requested a full tablet report, sending...
I20260809 08:52:32.882570 20325 ts_manager.cc:194] Registered new tserver with Master: a7984b066bfa4a7cbea4fd04bd72c6d8 (127.8.154.2:32787)
I20260809 08:52:32.882781 20035 ts_manager.cc:194] Registered new tserver with Master: a7984b066bfa4a7cbea4fd04bd72c6d8 (127.8.154.2:32787)
I20260809 08:52:32.883523 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.3:44751
--local_ip_for_outbound_sockets=127.8.154.3
--tserver_master_addrs=127.8.154.62:45607
--webserver_port=44923
--webserver_interface=127.8.154.3
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.3.keytab
--principal=oryx/127.8.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:45337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.8.154.62:45607,127.8.154.61:41867
--tserver_master_addrs=127.8.154.62:45607,127.8.154.61:41867,127.8.154.60:39223 with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:52:32.884615 19126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.845038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:45021 (local address 127.8.154.62:45607)
0809 08:52:32.845086 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:32.845088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:32.845098 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:52:32.845547 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:32.845549 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:32.845562 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:32.845597 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:32.846398 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.847148 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.878259 (+ 31111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.878600 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.878648 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.881295 (+ 2647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:32.881297 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:32.881298 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:32.881308 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:32.883340 (+ 2032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.883749 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.883750 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.883751 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.883808 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:32.884005 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:32.884006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:32.884007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:32.884190 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:32.884274 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:32.884450 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 08:52:32.884525 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 08:52:32.884631 20836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.844563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45607 (local address 127.8.154.2:45021)
0809 08:52:32.844743 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:32.844787 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:32.845448 (+ 661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:32.845703 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:32.845705 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:32.846244 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:32.846302 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.846305 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.847244 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.847244 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.878129 (+ 30885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.878134 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.878737 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.878738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.878819 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.879258 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:32.879264 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:32.881146 (+ 1882us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:32.883445 (+ 2299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:32.883446 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:32.883446 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:32.883643 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:32.883881 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:32.883881 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:32.883882 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:32.883928 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:32.884279 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:32.884280 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:32.884323 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 08:52:32.884406 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 08:52:32.884517 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":24,"thread_start_us":102,"threads_started":1}
I20260809 08:52:32.884930 20832 heartbeater.cc:347] Connected to a master server at 127.8.154.62:45607
I20260809 08:52:32.884989 20832 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:32.885063 20832 heartbeater.cc:511] Master 127.8.154.62:45607 requested a full tablet report, sending...
I20260809 08:52:32.885362 19049 ts_manager.cc:194] Re-registered known tserver with Master: a7984b066bfa4a7cbea4fd04bd72c6d8 (127.8.154.2:32787)
I20260809 08:52:32.885741 19049 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.154.2@KRBTEST.COM'} at 127.8.154.2:45021
I20260809 08:52:32.935539 20624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.932443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33806 (local address 127.8.154.60:39223)
0809 08:52:32.932525 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:32.932527 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:32.932538 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:52:32.932576 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:32.932593 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:32.932623 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:32.932690 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:32.932898 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.933796 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.934741 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.935186 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.935278 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.935361 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:32.935410 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 08:52:32.935450 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 08:52:32.935539 19971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:32.932354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:39223 (local address 127.0.0.1:33806)
0809 08:52:32.932421 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:32.932435 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:32.932492 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:32.932722 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:32.932724 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:32.932726 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:32.932805 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.932808 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.933890 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.933891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.934638 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:32.934640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:32.935281 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:32.935283 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:32.935349 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:32.935353 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:52:32.935405 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 08:52:32.935459 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 08:52:32.935918 20344 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260809 08:52:32.936071 20394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a22f631ee17d42f2a60289505b9f1fe4. Latest consensus state: current_term: 1 leader_uuid: "a22f631ee17d42f2a60289505b9f1fe4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: false } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: NON_VOTER last_known_addr { host: "127.8.154.60" port: 39223 } attrs { promote: true } } }
I20260809 08:52:32.936200 20394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [sys.catalog]: This master's current role is: LEARNER
W20260809 08:52:32.970649 20839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:32.970892 20839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:32.970935 20839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:32.972193 20839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:32.972262 20839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:32.972307 20839 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.3
I20260809 08:52:32.973867 20839 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:45337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.3.keytab
--principal=oryx/127.8.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.3:44751
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.154.3
--webserver_port=44923
--tserver_master_addrs=127.8.154.62:45607,127.8.154.61:41867,127.8.154.60:39223
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.3
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:32.974155 20839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:32.974501 20839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:32.980119 20848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:32.980293 20839 server_base.cc:1034] running on GCE node
W20260809 08:52:32.980193 20845 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:32.980125 20844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:32.980566 20839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:32.980792 20839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:32.981964 20839 hybrid_clock.cc:648] HybridClock initialized: now 1786265552981956 us; error 29 us; skew 500 ppm
Aug 09 08:52:32 dist-test-slave-t3q3 krb5kdc[18975](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265552, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:32.984630 20839 init.cc:377] Logged in from keytab as oryx/127.8.154.3@KRBTEST.COM (short username oryx)
I20260809 08:52:32.985255 20839 webserver.cc:460] Webserver started at http://127.8.154.3:44923/ using document root <none> and password file <none>
I20260809 08:52:32.985504 20839 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:32.985566 20839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:32.986994 20839 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:32.987674 20863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:32.987890 20839 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:32.988001 20839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-2/wal
uuid: "145b09aa53c74c73a021795b1fa0e19c"
format_stamp: "Formatted at 2026-08-09 08:52:31 on dist-test-slave-t3q3"
server_key: "71c44df4c62fc084b51d3638e15153b3"
server_key_iv: "e780284d8624427ed128f574f283b20b"
server_key_version: "encryptionkey@0"
I20260809 08:52:32.988444 20839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:33.007761 20839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:33.009858 20839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:33.010030 20839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:33.010311 20839 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:33.010646 20839 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:33.010701 20839 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:33.010749 20839 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:33.010777 20839 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:33.017329 20839 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.3:44751
I20260809 08:52:33.017370 21021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.3:44751 every 8 connection(s)
I20260809 08:52:33.017767 20839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
I20260809 08:52:33.019506 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 20839
Aug 09 08:52:33 dist-test-slave-t3q3 krb5kdc[18975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265552, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.3@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM
Aug 09 08:52:33 dist-test-slave-t3q3 krb5kdc[18975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265552, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.3@KRBTEST.COM for oryx/127.8.154.61@KRBTEST.COM
Aug 09 08:52:33 dist-test-slave-t3q3 krb5kdc[18975](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265552, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.3@KRBTEST.COM for oryx/127.8.154.60@KRBTEST.COM
I20260809 08:52:33.032305 21042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:33.021347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41144 (local address 127.8.154.61:41867)
0809 08:52:33.021487 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:33.021489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:33.021923 (+ 434us) server_negotiation.cc:407] Connection header received
0809 08:52:33.021948 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:33.021949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:33.021961 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:33.022011 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:33.022773 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.023532 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.024662 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.024933 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.025043 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:33.026601 (+ 1558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:33.026602 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:33.026602 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:33.026608 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:33.028530 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:33.029288 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:33.029289 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:33.029290 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:33.029384 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:33.031484 (+ 2100us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:33.031484 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:33.031485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:33.031597 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:33.031683 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:33.032126 (+ 443us) server_negotiation.cc:299] Negotiation successful
0809 08:52:33.032205 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":67,"threads_started":1}
I20260809 08:52:33.032382 21033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:33.018908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.61:41867 (local address 127.8.154.3:54675)
0809 08:52:33.019071 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:33.019082 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:33.020356 (+ 1274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:33.021462 (+ 1106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:33.021464 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:33.021619 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:33.021684 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:33.021686 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.022657 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.022657 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:33.023948 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:33.023951 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.024418 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.024419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:33.024563 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:33.025772 (+ 1209us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:33.025777 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:33.028178 (+ 2401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:33.030637 (+ 2459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:33.030645 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:33.030647 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:33.030949 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:33.031173 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:33.031174 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:33.031174 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:33.031317 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:33.031633 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:33.031640 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:33.031808 (+ 168us) client_negotiation.cc:769] Sending connection context
0809 08:52:33.031916 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 08:52:33.032138 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":61,"thread_start_us":69,"threads_started":1}
I20260809 08:52:33.032446 20624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:33.018634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:44609 (local address 127.8.154.60:39223)
0809 08:52:33.018675 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:33.018677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:33.018922 (+ 245us) server_negotiation.cc:407] Connection header received
0809 08:52:33.020475 (+ 1553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:33.020476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:33.020497 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:33.020530 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:33.021700 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.022524 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.024078 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.024414 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.024458 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:33.029228 (+ 4770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:33.029230 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:33.029230 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:33.029237 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:33.031034 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:33.031466 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:33.031467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:33.031468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:33.031525 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:33.031751 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:33.031752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:33.031753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:33.031911 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:33.031962 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:33.032236 (+ 274us) server_negotiation.cc:299] Negotiation successful
0809 08:52:33.032333 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 08:52:33.032713 20093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:33.018987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:54675 (local address 127.8.154.61:41867)
0809 08:52:33.019035 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:33.019037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:33.019137 (+ 100us) server_negotiation.cc:407] Connection header received
0809 08:52:33.020513 (+ 1376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:33.020514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:33.020527 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:33.020577 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:33.021760 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.022512 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.024028 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.024315 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.024406 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:33.028300 (+ 3894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:33.028301 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:33.028302 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:33.028308 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:33.030528 (+ 2220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:33.031039 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:33.031040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:33.031041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:33.031100 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:33.031393 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:33.031393 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:33.031394 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:33.031531 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:33.031604 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:33.032510 (+ 906us) server_negotiation.cc:299] Negotiation successful
0809 08:52:33.032600 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 08:52:33.033162 21029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:33.018579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:39223 (local address 127.8.154.3:44609)
0809 08:52:33.018777 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:33.018826 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:33.020355 (+ 1529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:33.020681 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:33.020682 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:33.021246 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:33.021574 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:33.021580 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.022660 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.022660 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:33.023952 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:33.023954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.024541 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.024541 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:33.024617 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:33.025364 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:33.025376 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:33.029119 (+ 3743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:33.031137 (+ 2018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:33.031138 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:33.031139 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:33.031376 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:33.031604 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:33.031605 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:33.031605 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:33.031671 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:33.032017 (+ 346us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:33.032018 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:33.032091 (+ 73us) client_negotiation.cc:769] Sending connection context
0809 08:52:33.032929 (+ 838us) client_negotiation.cc:240] Negotiation successful
0809 08:52:33.033054 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":44,"thread_start_us":95,"threads_started":1}
I20260809 08:52:33.033542 19126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:33.018573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:44159 (local address 127.8.154.62:45607)
0809 08:52:33.018632 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:33.018634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:33.018921 (+ 287us) server_negotiation.cc:407] Connection header received
0809 08:52:33.020484 (+ 1563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:33.020485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:33.020499 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:33.020533 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:33.021694 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.022512 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.024488 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.024795 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.024840 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:33.028882 (+ 4042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:33.028884 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:33.028884 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:33.028892 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:33.031214 (+ 2322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:33.032700 (+ 1486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:33.032701 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:33.032701 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:33.032774 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:33.033010 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:33.033011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:33.033011 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:33.033128 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:33.033210 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:33.033374 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 08:52:33.033454 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 08:52:33.033546 21030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:33.018579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45607 (local address 127.8.154.3:44159)
0809 08:52:33.018828 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:33.018838 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:33.020341 (+ 1503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:33.020670 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:33.020674 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:33.021239 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:33.021574 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:33.021579 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.022898 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.022899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:33.023949 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:33.023951 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.024921 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.024921 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:33.024996 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:33.025473 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:33.025479 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:33.028802 (+ 3323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:33.032396 (+ 3594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:33.032397 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:33.032398 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:33.032607 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:33.032851 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:33.032852 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:33.032852 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:33.032909 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:33.033203 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:33.033204 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:33.033248 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 08:52:33.033333 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 08:52:33.033440 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":34,"thread_start_us":100,"threads_started":1}
I20260809 08:52:33.033622 21022 heartbeater.cc:347] Connected to a master server at 127.8.154.60:39223
I20260809 08:52:33.033694 21027 heartbeater.cc:347] Connected to a master server at 127.8.154.61:41867
I20260809 08:52:33.033725 21022 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:33.033736 21027 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:33.033902 21024 heartbeater.cc:347] Connected to a master server at 127.8.154.62:45607
I20260809 08:52:33.033938 21024 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:33.033962 21022 heartbeater.cc:511] Master 127.8.154.60:39223 requested a full tablet report, sending...
I20260809 08:52:33.033962 21027 heartbeater.cc:511] Master 127.8.154.61:41867 requested a full tablet report, sending...
I20260809 08:52:33.033988 21024 heartbeater.cc:511] Master 127.8.154.62:45607 requested a full tablet report, sending...
I20260809 08:52:33.034422 19049 ts_manager.cc:194] Re-registered known tserver with Master: 145b09aa53c74c73a021795b1fa0e19c (127.8.154.3:44751)
I20260809 08:52:33.034446 20325 ts_manager.cc:194] Registered new tserver with Master: 145b09aa53c74c73a021795b1fa0e19c (127.8.154.3:44751)
I20260809 08:52:33.034478 20034 ts_manager.cc:194] Registered new tserver with Master: 145b09aa53c74c73a021795b1fa0e19c (127.8.154.3:44751)
I20260809 08:52:33.034829 19049 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.154.3@KRBTEST.COM'} at 127.8.154.3:44159
I20260809 08:52:33.036034 21041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:33.021145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54808 (local address 127.8.154.62:45607)
0809 08:52:33.025394 (+ 4249us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:33.025396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:33.025405 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:52:33.025431 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:33.025431 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:33.025444 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:33.026955 (+ 1511us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:33.027537 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.028432 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.029513 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.029810 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.029878 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:33.031399 (+ 1521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:33.031401 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:33.031401 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:33.031408 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:33.034584 (+ 3176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:33.035232 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:33.035233 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:33.035234 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:33.035301 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:33.035551 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:33.035552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:33.035552 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:33.035687 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:33.035778 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:33.035907 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 08:52:33.035969 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4213,"thread_start_us":109,"threads_started":1}
I20260809 08:52:33.037832 19049 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54808:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:52:33.040053 19049 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:52:33.040879 19972 sys_catalog.cc:455] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a22f631ee17d42f2a60289505b9f1fe4" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: false } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: NON_VOTER last_known_addr { host: "127.8.154.60" port: 39223 } attrs { promote: true } } }
I20260809 08:52:33.040966 19972 sys_catalog.cc:458] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:33.041724 21045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:33.027664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33822 (local address 127.8.154.60:39223)
0809 08:52:33.030210 (+ 2546us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:33.030212 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:33.030221 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:52:33.030244 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:33.030244 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:33.030255 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:33.030318 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:33.035767 (+ 5449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.036492 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.037211 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.037490 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.037564 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:33.038871 (+ 1307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:33.038872 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:33.038872 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:33.038879 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:33.040462 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:33.040877 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:33.040878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:33.040879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:33.040940 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:33.041157 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:33.041158 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:33.041159 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:33.041282 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:33.041388 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:33.041552 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 08:52:33.041641 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2506,"thread_start_us":57,"threads_started":1}
I20260809 08:52:33.042124 19974 raft_consensus.cc:1050] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4: attempting to promote NON_VOTER 88b956d3e2a24162ac5e1709d2832b3c to VOTER
I20260809 08:52:33.042268 19974 consensus_queue.cc:237] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: false } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 39223 } attrs { promote: false } }
I20260809 08:52:33.042642 20051 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [term 1 FOLLOWER]: Refusing update from remote peer a22f631ee17d42f2a60289505b9f1fe4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260809 08:52:33.042671 20344 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [term 1 LEARNER]: Refusing update from remote peer a22f631ee17d42f2a60289505b9f1fe4: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260809 08:52:33.042894 20283 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 39223 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 08:52:33.043018 19972 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 08:52:33.043784 20283 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 88b956d3e2a24162ac5e1709d2832b3c (127.8.154.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: false } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 39223 } attrs { promote: false } } }
I20260809 08:52:33.043921 20051 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 88b956d3e2a24162ac5e1709d2832b3c (127.8.154.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: false } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 39223 } attrs { promote: false } } }
I20260809 08:52:33.043928 20344 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 88b956d3e2a24162ac5e1709d2832b3c (127.8.154.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: false } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 39223 } attrs { promote: false } } }
I20260809 08:52:33.044834 20283 sys_catalog.cc:455] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a22f631ee17d42f2a60289505b9f1fe4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: false } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 39223 } attrs { promote: false } } }
I20260809 08:52:33.044920 20283 sys_catalog.cc:458] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:33.044945 20394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a22f631ee17d42f2a60289505b9f1fe4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: false } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 39223 } attrs { promote: false } } }
I20260809 08:52:33.044979 20283 sys_catalog.cc:455] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a22f631ee17d42f2a60289505b9f1fe4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: false } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 39223 } attrs { promote: false } } }
I20260809 08:52:33.045015 20394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:52:33.044960 20281 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a22f631ee17d42f2a60289505b9f1fe4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: false } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 39223 } attrs { promote: false } } }
I20260809 08:52:33.045024 20283 sys_catalog.cc:458] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:33.045048 20281 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:52:33.045061 20283 sys_catalog.cc:455] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a22f631ee17d42f2a60289505b9f1fe4" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: false } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 39223 } attrs { promote: false } } }
I20260809 08:52:33.045117 20283 sys_catalog.cc:458] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4 [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:33.050501 19971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:33.046562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:44029 (local address 127.0.0.1:59334)
0809 08:52:33.046874 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:33.046885 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:33.046931 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:33.047420 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:33.047420 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:33.047422 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:33.047457 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:33.047458 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.048710 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.048711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:33.049569 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:33.049571 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.050270 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.050271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:33.050331 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:33.050336 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:52:33.050400 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 08:52:33.050435 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292}
I20260809 08:52:33.050534 21064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:33.047029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59334 (local address 127.8.154.1:44029)
0809 08:52:33.047237 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:33.047242 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:33.047252 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:52:33.047291 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:33.047293 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:33.047308 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:33.047423 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:33.047540 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.048608 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.049659 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.050190 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.050270 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:33.050347 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:33.050415 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 08:52:33.050454 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":103,"threads_started":1}
I20260809 08:52:33.051487 21063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:33.046801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42378 (local address 127.8.154.3:44751)
0809 08:52:33.047027 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:33.047030 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:33.047041 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:52:33.047101 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:33.047103 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:33.047116 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:33.047235 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:33.047357 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.048877 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.050210 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.050922 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.051040 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:33.051118 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:33.051366 (+ 248us) server_negotiation.cc:299] Negotiation successful
0809 08:52:33.051410 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":97,"threads_started":1}
I20260809 08:52:33.051344 20504 tablet_service.cc:1467] Processing CreateTablet for tablet cf815ab86a0948cbb0e065e5f7e0b900 (DEFAULT_TABLE table=test-table [id=bb55cdf0e76f434286e439bb0c1439ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:33.051712 21060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:33.046666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:44751 (local address 127.0.0.1:42378)
0809 08:52:33.046982 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:33.046992 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:33.047034 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:33.047236 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:33.047237 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:33.047239 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:33.047288 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:33.047290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.048977 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.048978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:33.050116 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:33.050118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.051150 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.051151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:33.051227 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:33.051232 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:52:33.051287 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 08:52:33.051623 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":79,"threads_started":1}
I20260809 08:52:33.051739 20504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf815ab86a0948cbb0e065e5f7e0b900. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:33.051862 21065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:33.046562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:32787 (local address 127.0.0.1:55354)
0809 08:52:33.047546 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:33.047558 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:33.047654 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:33.047943 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:33.047944 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:33.047945 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:33.047985 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:33.047986 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.049234 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.049235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:33.050288 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:33.050290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.051537 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.051538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:33.051616 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:33.051620 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:52:33.051694 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 08:52:33.051744 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":961,"mutex_wait_us":620,"thread_start_us":69,"threads_started":1}
I20260809 08:52:33.052018 21062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:33.046708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55354 (local address 127.8.154.2:32787)
0809 08:52:33.046981 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:33.046988 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:33.047621 (+ 633us) server_negotiation.cc:407] Connection header received
0809 08:52:33.047780 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:33.047782 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:33.047800 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:33.047853 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:33.048082 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.049143 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.050379 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.051177 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.051218 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:33.051320 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:33.051718 (+ 398us) server_negotiation.cc:299] Negotiation successful
0809 08:52:33.051759 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":114,"threads_started":1}
I20260809 08:52:33.052752 20720 tablet_service.cc:1467] Processing CreateTablet for tablet cf815ab86a0948cbb0e065e5f7e0b900 (DEFAULT_TABLE table=test-table [id=bb55cdf0e76f434286e439bb0c1439ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:33.053045 20930 tablet_service.cc:1467] Processing CreateTablet for tablet cf815ab86a0948cbb0e065e5f7e0b900 (DEFAULT_TABLE table=test-table [id=bb55cdf0e76f434286e439bb0c1439ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:33.053243 20720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf815ab86a0948cbb0e065e5f7e0b900. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:33.053411 20930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf815ab86a0948cbb0e065e5f7e0b900. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:33.055805 21069 tablet_bootstrap.cc:492] T cf815ab86a0948cbb0e065e5f7e0b900 P ad6563f6630440b2afc6ba67e966bebf: Bootstrap starting.
I20260809 08:52:33.056556 21069 tablet_bootstrap.cc:654] T cf815ab86a0948cbb0e065e5f7e0b900 P ad6563f6630440b2afc6ba67e966bebf: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:33.056540 21070 tablet_bootstrap.cc:492] T cf815ab86a0948cbb0e065e5f7e0b900 P 145b09aa53c74c73a021795b1fa0e19c: Bootstrap starting.
I20260809 08:52:33.056995 21069 log.cc:824] T cf815ab86a0948cbb0e065e5f7e0b900 P ad6563f6630440b2afc6ba67e966bebf: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:33.057271 21070 tablet_bootstrap.cc:654] T cf815ab86a0948cbb0e065e5f7e0b900 P 145b09aa53c74c73a021795b1fa0e19c: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:33.057530 21071 tablet_bootstrap.cc:492] T cf815ab86a0948cbb0e065e5f7e0b900 P a7984b066bfa4a7cbea4fd04bd72c6d8: Bootstrap starting.
I20260809 08:52:33.057672 21070 log.cc:824] T cf815ab86a0948cbb0e065e5f7e0b900 P 145b09aa53c74c73a021795b1fa0e19c: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:33.058082 21069 tablet_bootstrap.cc:492] T cf815ab86a0948cbb0e065e5f7e0b900 P ad6563f6630440b2afc6ba67e966bebf: No bootstrap required, opened a new log
I20260809 08:52:33.058140 21069 ts_tablet_manager.cc:1397] T cf815ab86a0948cbb0e065e5f7e0b900 P ad6563f6630440b2afc6ba67e966bebf: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 08:52:33.058552 21070 tablet_bootstrap.cc:492] T cf815ab86a0948cbb0e065e5f7e0b900 P 145b09aa53c74c73a021795b1fa0e19c: No bootstrap required, opened a new log
I20260809 08:52:33.058625 21070 ts_tablet_manager.cc:1397] T cf815ab86a0948cbb0e065e5f7e0b900 P 145b09aa53c74c73a021795b1fa0e19c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:52:33.060263 21069 raft_consensus.cc:348] T cf815ab86a0948cbb0e065e5f7e0b900 P ad6563f6630440b2afc6ba67e966bebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad6563f6630440b2afc6ba67e966bebf" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 44029 } } peers { permanent_uuid: "a7984b066bfa4a7cbea4fd04bd72c6d8" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 32787 } } peers { permanent_uuid: "145b09aa53c74c73a021795b1fa0e19c" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 44751 } }
I20260809 08:52:33.060461 21069 raft_consensus.cc:374] T cf815ab86a0948cbb0e065e5f7e0b900 P ad6563f6630440b2afc6ba67e966bebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:33.060501 21069 raft_consensus.cc:729] T cf815ab86a0948cbb0e065e5f7e0b900 P ad6563f6630440b2afc6ba67e966bebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad6563f6630440b2afc6ba67e966bebf, State: Initialized, Role: FOLLOWER
I20260809 08:52:33.060617 21069 consensus_queue.cc:260] T cf815ab86a0948cbb0e065e5f7e0b900 P ad6563f6630440b2afc6ba67e966bebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad6563f6630440b2afc6ba67e966bebf" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 44029 } } peers { permanent_uuid: "a7984b066bfa4a7cbea4fd04bd72c6d8" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 32787 } } peers { permanent_uuid: "145b09aa53c74c73a021795b1fa0e19c" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 44751 } }
I20260809 08:52:33.060889 21069 ts_tablet_manager.cc:1428] T cf815ab86a0948cbb0e065e5f7e0b900 P ad6563f6630440b2afc6ba67e966bebf: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 08:52:33.060932 20615 heartbeater.cc:503] Master 127.8.154.62:45607 was elected leader, sending a full tablet report...
I20260809 08:52:33.061015 21070 raft_consensus.cc:348] T cf815ab86a0948cbb0e065e5f7e0b900 P 145b09aa53c74c73a021795b1fa0e19c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad6563f6630440b2afc6ba67e966bebf" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 44029 } } peers { permanent_uuid: "a7984b066bfa4a7cbea4fd04bd72c6d8" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 32787 } } peers { permanent_uuid: "145b09aa53c74c73a021795b1fa0e19c" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 44751 } }
I20260809 08:52:33.061167 21070 raft_consensus.cc:374] T cf815ab86a0948cbb0e065e5f7e0b900 P 145b09aa53c74c73a021795b1fa0e19c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:33.061218 21070 raft_consensus.cc:729] T cf815ab86a0948cbb0e065e5f7e0b900 P 145b09aa53c74c73a021795b1fa0e19c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145b09aa53c74c73a021795b1fa0e19c, State: Initialized, Role: FOLLOWER
I20260809 08:52:33.061349 21070 consensus_queue.cc:260] T cf815ab86a0948cbb0e065e5f7e0b900 P 145b09aa53c74c73a021795b1fa0e19c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad6563f6630440b2afc6ba67e966bebf" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 44029 } } peers { permanent_uuid: "a7984b066bfa4a7cbea4fd04bd72c6d8" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 32787 } } peers { permanent_uuid: "145b09aa53c74c73a021795b1fa0e19c" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 44751 } }
I20260809 08:52:33.061599 21070 ts_tablet_manager.cc:1428] T cf815ab86a0948cbb0e065e5f7e0b900 P 145b09aa53c74c73a021795b1fa0e19c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:52:33.062323 21024 heartbeater.cc:503] Master 127.8.154.62:45607 was elected leader, sending a full tablet report...
I20260809 08:52:33.062901 21071 tablet_bootstrap.cc:654] T cf815ab86a0948cbb0e065e5f7e0b900 P a7984b066bfa4a7cbea4fd04bd72c6d8: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:33.063405 21071 log.cc:824] T cf815ab86a0948cbb0e065e5f7e0b900 P a7984b066bfa4a7cbea4fd04bd72c6d8: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:33.064316 21071 tablet_bootstrap.cc:492] T cf815ab86a0948cbb0e065e5f7e0b900 P a7984b066bfa4a7cbea4fd04bd72c6d8: No bootstrap required, opened a new log
I20260809 08:52:33.064401 21071 ts_tablet_manager.cc:1397] T cf815ab86a0948cbb0e065e5f7e0b900 P a7984b066bfa4a7cbea4fd04bd72c6d8: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260809 08:52:33.065765 21071 raft_consensus.cc:348] T cf815ab86a0948cbb0e065e5f7e0b900 P a7984b066bfa4a7cbea4fd04bd72c6d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad6563f6630440b2afc6ba67e966bebf" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 44029 } } peers { permanent_uuid: "a7984b066bfa4a7cbea4fd04bd72c6d8" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 32787 } } peers { permanent_uuid: "145b09aa53c74c73a021795b1fa0e19c" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 44751 } }
I20260809 08:52:33.065909 21071 raft_consensus.cc:374] T cf815ab86a0948cbb0e065e5f7e0b900 P a7984b066bfa4a7cbea4fd04bd72c6d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:33.065958 21071 raft_consensus.cc:729] T cf815ab86a0948cbb0e065e5f7e0b900 P a7984b066bfa4a7cbea4fd04bd72c6d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7984b066bfa4a7cbea4fd04bd72c6d8, State: Initialized, Role: FOLLOWER
I20260809 08:52:33.066098 21071 consensus_queue.cc:260] T cf815ab86a0948cbb0e065e5f7e0b900 P a7984b066bfa4a7cbea4fd04bd72c6d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad6563f6630440b2afc6ba67e966bebf" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 44029 } } peers { permanent_uuid: "a7984b066bfa4a7cbea4fd04bd72c6d8" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 32787 } } peers { permanent_uuid: "145b09aa53c74c73a021795b1fa0e19c" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 44751 } }
I20260809 08:52:33.066342 21071 ts_tablet_manager.cc:1428] T cf815ab86a0948cbb0e065e5f7e0b900 P a7984b066bfa4a7cbea4fd04bd72c6d8: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:52:33.066413 20832 heartbeater.cc:503] Master 127.8.154.62:45607 was elected leader, sending a full tablet report...
W20260809 08:52:33.095016 20835 tablet.cc:2369] T cf815ab86a0948cbb0e065e5f7e0b900 P a7984b066bfa4a7cbea4fd04bd72c6d8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 08:52:33.171885 20618 tablet.cc:2369] T cf815ab86a0948cbb0e065e5f7e0b900 P ad6563f6630440b2afc6ba67e966bebf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 08:52:33.268724 21028 tablet.cc:2369] T cf815ab86a0948cbb0e065e5f7e0b900 P 145b09aa53c74c73a021795b1fa0e19c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:52:33.343343 21081 raft_consensus.cc:482] T cf815ab86a0948cbb0e065e5f7e0b900 P a7984b066bfa4a7cbea4fd04bd72c6d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:52:33.343479 21081 raft_consensus.cc:504] T cf815ab86a0948cbb0e065e5f7e0b900 P a7984b066bfa4a7cbea4fd04bd72c6d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad6563f6630440b2afc6ba67e966bebf" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 44029 } } peers { permanent_uuid: "a7984b066bfa4a7cbea4fd04bd72c6d8" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 32787 } } peers { permanent_uuid: "145b09aa53c74c73a021795b1fa0e19c" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 44751 } }
I20260809 08:52:33.343796 21081 leader_election.cc:288] T cf815ab86a0948cbb0e065e5f7e0b900 P a7984b066bfa4a7cbea4fd04bd72c6d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad6563f6630440b2afc6ba67e966bebf (127.8.154.1:44029), 145b09aa53c74c73a021795b1fa0e19c (127.8.154.3:44751)
I20260809 08:52:33.347137 21063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:33.344337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:58287 (local address 127.8.154.3:44751)
0809 08:52:33.344418 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:33.344420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:33.344430 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:52:33.344457 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:33.344458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:33.344469 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:33.344530 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:33.344849 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.345321 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.346382 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.346766 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.346853 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:33.346866 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:33.347026 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 08:52:33.347077 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 08:52:33.347121 20838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:33.344259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:44751 (local address 127.8.154.2:58287)
0809 08:52:33.344298 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:33.344306 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:33.344336 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:33.344674 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:33.344676 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:33.344678 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:33.344751 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:33.344753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.345414 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.345415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:33.346283 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:33.346285 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.346854 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.346854 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:33.346932 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:33.346935 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:52:33.347000 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 08:52:33.347045 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":51}
I20260809 08:52:33.347244 21064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:33.344210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:38837 (local address 127.8.154.1:44029)
0809 08:52:33.344304 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:33.344307 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:33.344323 (+ 16us) server_negotiation.cc:407] Connection header received
0809 08:52:33.344357 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:33.344358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:33.344371 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:33.344446 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:33.344674 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.345288 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.346417 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.346816 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.346909 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:33.346920 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:33.347114 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 08:52:33.347163 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 08:52:33.347276 20836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:33.344099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:44029 (local address 127.8.154.2:38837)
0809 08:52:33.344164 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:33.344177 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:33.344229 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:33.344453 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:33.344454 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:33.344455 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:33.344559 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:33.344562 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.345441 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.345443 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:33.346306 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:33.346308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.346923 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.346925 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:33.347015 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:33.347019 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:52:33.347087 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 08:52:33.347131 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":23}
I20260809 08:52:33.347534 20956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf815ab86a0948cbb0e065e5f7e0b900" candidate_uuid: "a7984b066bfa4a7cbea4fd04bd72c6d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145b09aa53c74c73a021795b1fa0e19c" is_pre_election: true
I20260809 08:52:33.347608 20535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf815ab86a0948cbb0e065e5f7e0b900" candidate_uuid: "a7984b066bfa4a7cbea4fd04bd72c6d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad6563f6630440b2afc6ba67e966bebf" is_pre_election: true
I20260809 08:52:33.347713 20956 raft_consensus.cc:2454] T cf815ab86a0948cbb0e065e5f7e0b900 P 145b09aa53c74c73a021795b1fa0e19c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7984b066bfa4a7cbea4fd04bd72c6d8 in term 0.
I20260809 08:52:33.347761 20535 raft_consensus.cc:2454] T cf815ab86a0948cbb0e065e5f7e0b900 P ad6563f6630440b2afc6ba67e966bebf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7984b066bfa4a7cbea4fd04bd72c6d8 in term 0.
I20260809 08:52:33.347973 20653 leader_election.cc:302] T cf815ab86a0948cbb0e065e5f7e0b900 P a7984b066bfa4a7cbea4fd04bd72c6d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145b09aa53c74c73a021795b1fa0e19c, a7984b066bfa4a7cbea4fd04bd72c6d8; no voters:
I20260809 08:52:33.348140 21081 raft_consensus.cc:2781] T cf815ab86a0948cbb0e065e5f7e0b900 P a7984b066bfa4a7cbea4fd04bd72c6d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:52:33.348206 21081 raft_consensus.cc:482] T cf815ab86a0948cbb0e065e5f7e0b900 P a7984b066bfa4a7cbea4fd04bd72c6d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:52:33.348248 21081 raft_consensus.cc:3037] T cf815ab86a0948cbb0e065e5f7e0b900 P a7984b066bfa4a7cbea4fd04bd72c6d8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:33.349097 21081 raft_consensus.cc:504] T cf815ab86a0948cbb0e065e5f7e0b900 P a7984b066bfa4a7cbea4fd04bd72c6d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad6563f6630440b2afc6ba67e966bebf" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 44029 } } peers { permanent_uuid: "a7984b066bfa4a7cbea4fd04bd72c6d8" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 32787 } } peers { permanent_uuid: "145b09aa53c74c73a021795b1fa0e19c" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 44751 } }
I20260809 08:52:33.349243 21081 leader_election.cc:288] T cf815ab86a0948cbb0e065e5f7e0b900 P a7984b066bfa4a7cbea4fd04bd72c6d8 [CANDIDATE]: Term 1 election: Requested vote from peers ad6563f6630440b2afc6ba67e966bebf (127.8.154.1:44029), 145b09aa53c74c73a021795b1fa0e19c (127.8.154.3:44751)
I20260809 08:52:33.349457 20535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf815ab86a0948cbb0e065e5f7e0b900" candidate_uuid: "a7984b066bfa4a7cbea4fd04bd72c6d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad6563f6630440b2afc6ba67e966bebf"
I20260809 08:52:33.349442 20956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf815ab86a0948cbb0e065e5f7e0b900" candidate_uuid: "a7984b066bfa4a7cbea4fd04bd72c6d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145b09aa53c74c73a021795b1fa0e19c"
I20260809 08:52:33.349536 20535 raft_consensus.cc:3037] T cf815ab86a0948cbb0e065e5f7e0b900 P ad6563f6630440b2afc6ba67e966bebf [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:33.349567 20956 raft_consensus.cc:3037] T cf815ab86a0948cbb0e065e5f7e0b900 P 145b09aa53c74c73a021795b1fa0e19c [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:33.350237 20535 raft_consensus.cc:2454] T cf815ab86a0948cbb0e065e5f7e0b900 P ad6563f6630440b2afc6ba67e966bebf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7984b066bfa4a7cbea4fd04bd72c6d8 in term 1.
I20260809 08:52:33.350423 20956 raft_consensus.cc:2454] T cf815ab86a0948cbb0e065e5f7e0b900 P 145b09aa53c74c73a021795b1fa0e19c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7984b066bfa4a7cbea4fd04bd72c6d8 in term 1.
I20260809 08:52:33.350420 20653 leader_election.cc:302] T cf815ab86a0948cbb0e065e5f7e0b900 P a7984b066bfa4a7cbea4fd04bd72c6d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7984b066bfa4a7cbea4fd04bd72c6d8, ad6563f6630440b2afc6ba67e966bebf; no voters:
I20260809 08:52:33.350520 21081 raft_consensus.cc:2781] T cf815ab86a0948cbb0e065e5f7e0b900 P a7984b066bfa4a7cbea4fd04bd72c6d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:52:33.350656 21081 raft_consensus.cc:686] T cf815ab86a0948cbb0e065e5f7e0b900 P a7984b066bfa4a7cbea4fd04bd72c6d8 [term 1 LEADER]: Becoming Leader. State: Replica: a7984b066bfa4a7cbea4fd04bd72c6d8, State: Running, Role: LEADER
I20260809 08:52:33.350768 21081 consensus_queue.cc:237] T cf815ab86a0948cbb0e065e5f7e0b900 P a7984b066bfa4a7cbea4fd04bd72c6d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad6563f6630440b2afc6ba67e966bebf" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 44029 } } peers { permanent_uuid: "a7984b066bfa4a7cbea4fd04bd72c6d8" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 32787 } } peers { permanent_uuid: "145b09aa53c74c73a021795b1fa0e19c" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 44751 } }
I20260809 08:52:33.351787 19047 catalog_manager.cc:5495] T cf815ab86a0948cbb0e065e5f7e0b900 P a7984b066bfa4a7cbea4fd04bd72c6d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a7984b066bfa4a7cbea4fd04bd72c6d8 (127.8.154.2). New cstate: current_term: 1 leader_uuid: "a7984b066bfa4a7cbea4fd04bd72c6d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad6563f6630440b2afc6ba67e966bebf" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 44029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7984b066bfa4a7cbea4fd04bd72c6d8" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 32787 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "145b09aa53c74c73a021795b1fa0e19c" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 44751 } health_report { overall_health: UNKNOWN } } }
I20260809 08:52:33.407419 21062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:33.404741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55366 (local address 127.8.154.2:32787)
0809 08:52:33.404807 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:33.404809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:33.404818 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:52:33.404866 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:33.404867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:33.404889 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:33.404956 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:52:33.405142 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.405624 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.406526 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.406807 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.406876 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:33.407082 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:52:33.407188 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:52:33.407255 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:33.407336 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 08:52:33.407357 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 08:52:33.409752 20535 raft_consensus.cc:1261] T cf815ab86a0948cbb0e065e5f7e0b900 P ad6563f6630440b2afc6ba67e966bebf [term 1 FOLLOWER]: Refusing update from remote peer a7984b066bfa4a7cbea4fd04bd72c6d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:52:33.409780 20956 raft_consensus.cc:1261] T cf815ab86a0948cbb0e065e5f7e0b900 P 145b09aa53c74c73a021795b1fa0e19c [term 1 FOLLOWER]: Refusing update from remote peer a7984b066bfa4a7cbea4fd04bd72c6d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:52:33.410131 21081 consensus_queue.cc:1035] T cf815ab86a0948cbb0e065e5f7e0b900 P a7984b066bfa4a7cbea4fd04bd72c6d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad6563f6630440b2afc6ba67e966bebf" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 44029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:52:33.410327 21084 consensus_queue.cc:1035] T cf815ab86a0948cbb0e065e5f7e0b900 P a7984b066bfa4a7cbea4fd04bd72c6d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "145b09aa53c74c73a021795b1fa0e19c" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 44751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:52:33.414016 21090 mvcc.cc:205] Tried to move back new op lower bound from 7316543706762956800 to 7316543706525429760. Current Snapshot: MvccSnapshot[applied={T|T < 7316543706762956800}]
I20260809 08:52:33.416028 21092 mvcc.cc:205] Tried to move back new op lower bound from 7316543706762956800 to 7316543706525429760. Current Snapshot: MvccSnapshot[applied={T|T < 7316543706762956800}]
I20260809 08:52:33.418135 19049 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:52:33.418253 19049 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:52:33.419997 19049 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a22f631ee17d42f2a60289505b9f1fe4: Sending DeleteTablet for 3 replicas of tablet cf815ab86a0948cbb0e065e5f7e0b900
I20260809 08:52:33.420512 20720 tablet_service.cc:1514] Processing DeleteTablet for tablet cf815ab86a0948cbb0e065e5f7e0b900 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:33 UTC) from {username='oryx', principal='oryx/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:55354
I20260809 08:52:33.420873 21104 tablet_replica.cc:330] T cf815ab86a0948cbb0e065e5f7e0b900 P a7984b066bfa4a7cbea4fd04bd72c6d8: stopping tablet replica
I20260809 08:52:33.420940 20504 tablet_service.cc:1514] Processing DeleteTablet for tablet cf815ab86a0948cbb0e065e5f7e0b900 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:33 UTC) from {username='oryx', principal='oryx/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:59334
I20260809 08:52:33.421275 21104 raft_consensus.cc:2229] T cf815ab86a0948cbb0e065e5f7e0b900 P a7984b066bfa4a7cbea4fd04bd72c6d8 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:52:33.421247 20930 tablet_service.cc:1514] Processing DeleteTablet for tablet cf815ab86a0948cbb0e065e5f7e0b900 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:33 UTC) from {username='oryx', principal='oryx/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:42378
I20260809 08:52:33.421373 21107 tablet_replica.cc:330] T cf815ab86a0948cbb0e065e5f7e0b900 P ad6563f6630440b2afc6ba67e966bebf: stopping tablet replica
I20260809 08:52:33.421551 21107 raft_consensus.cc:2229] T cf815ab86a0948cbb0e065e5f7e0b900 P ad6563f6630440b2afc6ba67e966bebf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:52:33.421571 21104 raft_consensus.cc:2258] T cf815ab86a0948cbb0e065e5f7e0b900 P a7984b066bfa4a7cbea4fd04bd72c6d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:52:33.421674 21107 raft_consensus.cc:2258] T cf815ab86a0948cbb0e065e5f7e0b900 P ad6563f6630440b2afc6ba67e966bebf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:52:33.422140 21107 ts_tablet_manager.cc:1905] T cf815ab86a0948cbb0e065e5f7e0b900 P ad6563f6630440b2afc6ba67e966bebf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:52:33.422387 21104 ts_tablet_manager.cc:1905] T cf815ab86a0948cbb0e065e5f7e0b900 P a7984b066bfa4a7cbea4fd04bd72c6d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:52:33.422520 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 19000
I20260809 08:52:33.423856 21107 ts_tablet_manager.cc:1918] T cf815ab86a0948cbb0e065e5f7e0b900 P ad6563f6630440b2afc6ba67e966bebf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:52:33.423925 21107 log.cc:1196] T cf815ab86a0948cbb0e065e5f7e0b900 P ad6563f6630440b2afc6ba67e966bebf: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-0/wal/wals/cf815ab86a0948cbb0e065e5f7e0b900
I20260809 08:52:33.424181 21107 ts_tablet_manager.cc:1939] T cf815ab86a0948cbb0e065e5f7e0b900 P ad6563f6630440b2afc6ba67e966bebf: Deleting consensus metadata
I20260809 08:52:33.425470 21104 ts_tablet_manager.cc:1918] T cf815ab86a0948cbb0e065e5f7e0b900 P a7984b066bfa4a7cbea4fd04bd72c6d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:52:33.425591 21104 log.cc:1196] T cf815ab86a0948cbb0e065e5f7e0b900 P a7984b066bfa4a7cbea4fd04bd72c6d8: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-1/wal/wals/cf815ab86a0948cbb0e065e5f7e0b900
I20260809 08:52:33.425945 21104 ts_tablet_manager.cc:1939] T cf815ab86a0948cbb0e065e5f7e0b900 P a7984b066bfa4a7cbea4fd04bd72c6d8: Deleting consensus metadata
I20260809 08:52:33.426455 21108 tablet_replica.cc:330] T cf815ab86a0948cbb0e065e5f7e0b900 P 145b09aa53c74c73a021795b1fa0e19c: stopping tablet replica
I20260809 08:52:33.426620 21108 raft_consensus.cc:2229] T cf815ab86a0948cbb0e065e5f7e0b900 P 145b09aa53c74c73a021795b1fa0e19c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:52:33.426788 21108 raft_consensus.cc:2258] T cf815ab86a0948cbb0e065e5f7e0b900 P 145b09aa53c74c73a021795b1fa0e19c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:52:33.432415 21108 ts_tablet_manager.cc:1905] T cf815ab86a0948cbb0e065e5f7e0b900 P 145b09aa53c74c73a021795b1fa0e19c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:52:33.434193 21108 ts_tablet_manager.cc:1918] T cf815ab86a0948cbb0e065e5f7e0b900 P 145b09aa53c74c73a021795b1fa0e19c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:52:33.434293 21108 log.cc:1196] T cf815ab86a0948cbb0e065e5f7e0b900 P 145b09aa53c74c73a021795b1fa0e19c: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-2/wal/wals/cf815ab86a0948cbb0e065e5f7e0b900
I20260809 08:52:33.434567 21108 ts_tablet_manager.cc:1939] T cf815ab86a0948cbb0e065e5f7e0b900 P 145b09aa53c74c73a021795b1fa0e19c: Deleting consensus metadata
W20260809 08:52:33.442042 20869 connection.cc:535] server connection from 127.0.0.1:42378 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 08:52:33.442325 20652 connection.cc:535] server connection from 127.0.0.1:55354 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260809 08:52:33.442366 20433 connection.cc:535] server connection from 127.0.0.1:59334 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 08:52:33.443732 21045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:33.421175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33834 (local address 127.8.154.60:39223)
0809 08:52:33.421225 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:33.421228 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:33.435060 (+ 13832us) server_negotiation.cc:407] Connection header received
0809 08:52:33.435104 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:33.435105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:33.435123 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:33.435187 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:33.435947 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.436887 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.438015 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.438410 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.438512 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:33.440200 (+ 1688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:33.440201 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:33.440202 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:33.440211 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:33.442281 (+ 2070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:33.442771 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:33.442772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:33.442773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:33.442846 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:33.443084 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:33.443085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:33.443086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:33.443236 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:33.443316 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:33.443537 (+ 221us) server_negotiation.cc:299] Negotiation successful
0809 08:52:33.443631 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 08:52:33.453632 21045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:33.444647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33842 (local address 127.8.154.60:39223)
0809 08:52:33.444685 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:33.444687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:33.445045 (+ 358us) server_negotiation.cc:407] Connection header received
0809 08:52:33.445110 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:33.445111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:33.445122 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:33.445185 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:33.446489 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.447247 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.448276 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.448534 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.448612 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:33.450030 (+ 1418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:33.450031 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:33.450032 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:33.450040 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:33.452280 (+ 2240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:33.452717 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:33.452718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:33.452719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:33.452788 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:33.453074 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:33.453075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:33.453075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:33.453218 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:33.453316 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:33.453451 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 08:52:33.453530 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 08:52:33.453708 20093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:33.444780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41146 (local address 127.8.154.61:41867)
0809 08:52:33.444835 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:33.444837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:33.445087 (+ 250us) server_negotiation.cc:407] Connection header received
0809 08:52:33.445191 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:33.445192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:33.445204 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:33.445244 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:33.445900 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.446722 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.447763 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:33.448157 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:33.448238 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:33.449823 (+ 1585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:33.449824 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:33.449825 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:33.449834 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:33.452043 (+ 2209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:33.452628 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:33.452630 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:33.452630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:33.452697 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:33.452961 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:33.452962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:33.452963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:33.453139 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:33.453210 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:33.453468 (+ 258us) server_negotiation.cc:299] Negotiation successful
0809 08:52:33.453550 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 08:52:34.415670 21118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:34.415193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45607 (local address 127.8.154.2:40173)
0809 08:52:34.415475 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:34.415609 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:45607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":72,"threads_started":1}
W20260809 08:52:34.415951 20832 heartbeater.cc:644] Failed to heartbeat to 127.8.154.62:45607 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.154.62:45607: connect: Connection refused (error 111)
I20260809 08:52:34.416286 21119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:34.416082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45607 (local address 127.8.154.1:57813)
0809 08:52:34.416175 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:34.416249 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:45607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78,"thread_start_us":53,"threads_started":1}
W20260809 08:52:34.416497 20615 heartbeater.cc:644] Failed to heartbeat to 127.8.154.62:45607 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.154.62:45607: connect: Connection refused (error 111)
I20260809 08:52:34.417095 21120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:34.416867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45607 (local address 127.8.154.3:45679)
0809 08:52:34.416992 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:34.417050 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:45607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":55,"threads_started":1}
W20260809 08:52:34.417286 21024 heartbeater.cc:644] Failed to heartbeat to 127.8.154.62:45607 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.8.154.62:45607: connect: Connection refused (error 111)
I20260809 08:52:34.920576 21121 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a22f631ee17d42f2a60289505b9f1fe4)
I20260809 08:52:34.920939 21121 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: false } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 39223 } attrs { promote: false } }
I20260809 08:52:34.921319 21121 leader_election.cc:288] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a22f631ee17d42f2a60289505b9f1fe4 (127.8.154.62:45607), 88b956d3e2a24162ac5e1709d2832b3c (127.8.154.60:39223)
I20260809 08:52:34.921703 21122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:34.921452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45607 (local address 127.0.0.1:55314)
0809 08:52:34.921599 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:34.921661 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:45607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":85,"threads_started":1}
W20260809 08:52:34.921875 20014 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.154.62:45607: connect: Connection refused (error 111)
I20260809 08:52:34.922046 21126 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a22f631ee17d42f2a60289505b9f1fe4)
I20260809 08:52:34.922197 21126 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: false } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 39223 } attrs { promote: false } }
I20260809 08:52:34.922386 21122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:34.922238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45607 (local address 127.0.0.1:55328)
0809 08:52:34.922291 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:34.922325 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:45607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 08:52:34.922470 20014 leader_election.cc:334] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a22f631ee17d42f2a60289505b9f1fe4 (127.8.154.62:45607): Network error: Client connection negotiation failed: client connection to 127.8.154.62:45607: connect: Connection refused (error 111)
I20260809 08:52:34.922773 21126 leader_election.cc:288] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a22f631ee17d42f2a60289505b9f1fe4 (127.8.154.62:45607), a9f86b88da034484ab3f19ad3b945ade (127.8.154.61:41867)
I20260809 08:52:34.922827 21129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:34.922641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45607 (local address 127.0.0.1:55336)
0809 08:52:34.922749 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:34.922794 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:45607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":64,"threads_started":1}
W20260809 08:52:34.922997 20311 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.154.62:45607: connect: Connection refused (error 111)
I20260809 08:52:34.923480 21132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:34.923282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45607 (local address 127.0.0.1:55350)
0809 08:52:34.923414 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:34.923448 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:45607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":80,"threads_started":1}
W20260809 08:52:34.923588 20311 leader_election.cc:334] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a22f631ee17d42f2a60289505b9f1fe4 (127.8.154.62:45607): Network error: Client connection negotiation failed: client connection to 127.8.154.62:45607: connect: Connection refused (error 111)
I20260809 08:52:34.925540 21123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:34.921530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:39223 (local address 127.0.0.1:33844)
0809 08:52:34.921774 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:34.921786 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:34.922278 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:34.922449 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:34.922450 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:34.922452 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:34.922515 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:34.922518 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:34.923626 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:34.923627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:34.924695 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:34.924698 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:34.925345 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:34.925346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:34.925402 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:34.925406 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:52:34.925459 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 08:52:34.925494 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":73,"threads_started":1}
I20260809 08:52:34.925582 21124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:34.921666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33844 (local address 127.8.154.60:39223)
0809 08:52:34.921785 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:34.921788 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:34.922145 (+ 357us) server_negotiation.cc:407] Connection header received
0809 08:52:34.922360 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:34.922361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:34.922373 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:34.922443 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:34.922577 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:34.923524 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:34.924805 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:34.925263 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:34.925340 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:34.925353 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:34.925480 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 08:52:34.925516 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":97,"thread_start_us":76,"threads_started":1}
I20260809 08:52:34.925892 20344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9f86b88da034484ab3f19ad3b945ade" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "88b956d3e2a24162ac5e1709d2832b3c" is_pre_election: true
I20260809 08:52:34.926010 20344 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9f86b88da034484ab3f19ad3b945ade in term 1.
I20260809 08:52:34.926075 21130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:34.922955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41150 (local address 127.8.154.61:41867)
0809 08:52:34.923119 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:34.923121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:34.923127 (+ 6us) server_negotiation.cc:407] Connection header received
0809 08:52:34.923151 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:34.923152 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:34.923162 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:34.923242 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:34.923423 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:34.924178 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:34.925163 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:34.925763 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:34.925841 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:34.925852 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:34.925968 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 08:52:34.926010 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":95,"threads_started":1}
I20260809 08:52:34.926110 21129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:34.922914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.61:41867 (local address 127.0.0.1:41150)
0809 08:52:34.922943 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:34.922953 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:34.923014 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:34.923293 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:34.923294 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:34.923298 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:34.923352 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:34.923354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:34.924260 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:34.924261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:34.925083 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:34.925086 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:34.925838 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:34.925838 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:34.925892 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:34.925904 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 08:52:34.925947 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 08:52:34.925979 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18,"mutex_wait_us":2}
I20260809 08:52:34.926177 20018 leader_election.cc:302] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 88b956d3e2a24162ac5e1709d2832b3c, a9f86b88da034484ab3f19ad3b945ade; no voters: a22f631ee17d42f2a60289505b9f1fe4
I20260809 08:52:34.926290 21121 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 08:52:34.926331 21121 raft_consensus.cc:482] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [term 1 FOLLOWER]: Starting leader election (detected failure of leader a22f631ee17d42f2a60289505b9f1fe4)
I20260809 08:52:34.926357 21121 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [term 1 FOLLOWER]: Advancing to term 2
I20260809 08:52:34.926369 20051 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "88b956d3e2a24162ac5e1709d2832b3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a9f86b88da034484ab3f19ad3b945ade" is_pre_election: true
I20260809 08:52:34.927103 21121 raft_consensus.cc:504] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: false } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 39223 } attrs { promote: false } }
I20260809 08:52:34.927264 20051 raft_consensus.cc:2379] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 88b956d3e2a24162ac5e1709d2832b3c in current term 2: Already voted for candidate a9f86b88da034484ab3f19ad3b945ade in this term.
I20260809 08:52:34.927290 21121 leader_election.cc:288] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [CANDIDATE]: Term 2 election: Requested vote from peers a22f631ee17d42f2a60289505b9f1fe4 (127.8.154.62:45607), 88b956d3e2a24162ac5e1709d2832b3c (127.8.154.60:39223)
I20260809 08:52:34.927440 21123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:34.927358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45607 (local address 127.0.0.1:55362)
0809 08:52:34.927392 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:34.927419 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:45607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260809 08:52:34.927441 20311 leader_election.cc:302] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 88b956d3e2a24162ac5e1709d2832b3c; no voters: a22f631ee17d42f2a60289505b9f1fe4, a9f86b88da034484ab3f19ad3b945ade
I20260809 08:52:34.927553 21126 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [term 1 FOLLOWER]: Advancing to term 2
I20260809 08:52:34.927598 20344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a9f86b88da034484ab3f19ad3b945ade" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "88b956d3e2a24162ac5e1709d2832b3c"
I20260809 08:52:34.927763 21123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:34.927674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45607 (local address 127.0.0.1:55372)
0809 08:52:34.927708 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:34.927737 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:45607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260809 08:52:34.927873 20014 leader_election.cc:334] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a22f631ee17d42f2a60289505b9f1fe4 (127.8.154.62:45607): Network error: Client connection negotiation failed: client connection to 127.8.154.62:45607: connect: Connection refused (error 111)
I20260809 08:52:34.928421 21126 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 08:52:34.929005 20344 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9f86b88da034484ab3f19ad3b945ade in term 2.
I20260809 08:52:34.929184 20018 leader_election.cc:302] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 88b956d3e2a24162ac5e1709d2832b3c, a9f86b88da034484ab3f19ad3b945ade; no voters: a22f631ee17d42f2a60289505b9f1fe4
I20260809 08:52:34.929287 21121 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [term 2 FOLLOWER]: Leader election won for term 2
I20260809 08:52:34.929433 21121 raft_consensus.cc:686] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [term 2 LEADER]: Becoming Leader. State: Replica: a9f86b88da034484ab3f19ad3b945ade, State: Running, Role: LEADER
I20260809 08:52:34.929517 21121 consensus_queue.cc:237] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: false } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 39223 } attrs { promote: false } }
I20260809 08:52:34.929889 21133 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9f86b88da034484ab3f19ad3b945ade. Latest consensus state: current_term: 2 leader_uuid: "a9f86b88da034484ab3f19ad3b945ade" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: false } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 39223 } attrs { promote: false } } }
I20260809 08:52:34.929997 21133 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:34.930174 21136 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:52:34.930681 21136 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:52:34.930773 21136 catalog_manager.cc:1259] Loaded cluster ID: ac149d96c0c84228a4d9a3175bba4c67
I20260809 08:52:34.930809 21136 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:52:34.930971 21136 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:52:34.931059 21136 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade: Loaded TSK: 0
I20260809 08:52:34.931214 21136 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:52:34.940137 20034 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41146:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:52:34.941730 20034 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:52:34.942765 20344 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [term 2 FOLLOWER]: Refusing update from remote peer a9f86b88da034484ab3f19ad3b945ade: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260809 08:52:34.942786 21123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:34.942632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45607 (local address 127.0.0.1:55388)
0809 08:52:34.942708 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:34.942751 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:45607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 08:52:34.943058 21133 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [LEADER]: Connected to new peer: Peer: permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 39223 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260809 08:52:34.943156 21123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:34.943047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45607 (local address 127.0.0.1:55394)
0809 08:52:34.943104 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:34.943133 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:45607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 08:52:34.943339 20014 consensus_peers.cc:487] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade -> Peer a22f631ee17d42f2a60289505b9f1fe4 (127.8.154.62:45607): Couldn't send request to peer a22f631ee17d42f2a60289505b9f1fe4. Status: Network error: Client connection negotiation failed: client connection to 127.8.154.62:45607: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 08:52:34.943785 21126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a9f86b88da034484ab3f19ad3b945ade. Latest consensus state: current_term: 2 leader_uuid: "a9f86b88da034484ab3f19ad3b945ade" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: false } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 39223 } attrs { promote: false } } }
I20260809 08:52:34.943910 21126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:52:34.944353 21138 mvcc.cc:205] Tried to move back new op lower bound from 7316543713043427328 to 7316543712992014336. Current Snapshot: MvccSnapshot[applied={T|T < 7316543713043427328}]
I20260809 08:52:34.944548 21126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "a9f86b88da034484ab3f19ad3b945ade" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: false } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 39223 } attrs { promote: false } } }
I20260809 08:52:34.944641 21126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88b956d3e2a24162ac5e1709d2832b3c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:52:34.944730 21121 sys_catalog.cc:455] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "a9f86b88da034484ab3f19ad3b945ade" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "a22f631ee17d42f2a60289505b9f1fe4" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45607 } } peers { permanent_uuid: "a9f86b88da034484ab3f19ad3b945ade" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 41867 } attrs { promote: false } } peers { permanent_uuid: "88b956d3e2a24162ac5e1709d2832b3c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 39223 } attrs { promote: false } } }
I20260809 08:52:34.944813 21121 sys_catalog.cc:458] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:34.950163 21122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:34.946453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:44029 (local address 127.0.0.1:59338)
0809 08:52:34.946646 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:34.946658 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:34.946732 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:34.946931 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:34.946932 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:34.946935 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:34.946991 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:34.946994 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:34.947991 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:34.947992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:34.949303 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:34.949305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:34.949895 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:34.949896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:34.949962 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:34.949966 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:52:34.950042 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 08:52:34.950090 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":47}
I20260809 08:52:34.950166 21145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:34.946540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59338 (local address 127.8.154.1:44029)
0809 08:52:34.946667 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:34.946669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:34.946706 (+ 37us) server_negotiation.cc:407] Connection header received
0809 08:52:34.946802 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:34.946802 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:34.946814 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:34.946876 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:34.947075 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:34.947855 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:34.949401 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:34.949787 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:34.949890 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:34.949903 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:34.950052 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 08:52:34.950097 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":70,"threads_started":1}
I20260809 08:52:34.950398 21123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:34.946550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:44751 (local address 127.0.0.1:42386)
0809 08:52:34.946687 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:34.946698 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:34.946739 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:34.947071 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:34.947072 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:34.947074 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:34.947116 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:34.947118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:34.947991 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:34.947992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:34.949281 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:34.949285 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:34.950083 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:34.950084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:34.950146 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:34.950153 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 08:52:34.950183 (+ 30us) client_negotiation.cc:240] Negotiation successful
0809 08:52:34.950331 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":75}
I20260809 08:52:34.950469 20504 tablet_service.cc:1467] Processing CreateTablet for tablet 1657de8c2c184559b684c9ad9e23346e (DEFAULT_TABLE table=test-table [id=82f8cab4c5e64d57afcdb69ed1586d71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:34.950641 20504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1657de8c2c184559b684c9ad9e23346e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:34.950754 21148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:34.946847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42386 (local address 127.8.154.3:44751)
0809 08:52:34.946961 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:34.946963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:34.946970 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:52:34.946994 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:34.946995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:34.947005 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:34.947069 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:34.947171 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:34.947891 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:34.949385 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:34.949982 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:34.950588 (+ 606us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:34.950606 (+ 18us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:34.950627 (+ 21us) server_negotiation.cc:299] Negotiation successful
0809 08:52:34.950672 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":60,"threads_started":1}
I20260809 08:52:34.950951 20930 tablet_service.cc:1467] Processing CreateTablet for tablet 1657de8c2c184559b684c9ad9e23346e (DEFAULT_TABLE table=test-table [id=82f8cab4c5e64d57afcdb69ed1586d71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:34.951089 20930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1657de8c2c184559b684c9ad9e23346e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:34.951238 21146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:34.946555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:32787 (local address 127.0.0.1:55372)
0809 08:52:34.946744 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:34.946754 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:34.947995 (+ 1241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:34.948287 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:34.948288 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:34.948290 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:34.948335 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:34.948337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:34.949035 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:34.949036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:34.950112 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:34.950114 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:34.950977 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:34.950978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:34.951040 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:34.951045 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:52:34.951104 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 08:52:34.951148 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":5,"thread_start_us":89,"threads_started":1}
I20260809 08:52:34.951531 21147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:34.946625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55372 (local address 127.8.154.2:32787)
0809 08:52:34.948149 (+ 1524us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:34.948151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:34.948159 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:52:34.948187 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:34.948188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:34.948197 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:34.948270 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:34.948414 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:34.948945 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:34.950204 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:34.950859 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:34.950928 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:34.950941 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:34.951113 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 08:52:34.951205 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1500,"thread_start_us":54,"threads_started":1}
I20260809 08:52:34.951886 20720 tablet_service.cc:1467] Processing CreateTablet for tablet 1657de8c2c184559b684c9ad9e23346e (DEFAULT_TABLE table=test-table [id=82f8cab4c5e64d57afcdb69ed1586d71]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:34.951999 20720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1657de8c2c184559b684c9ad9e23346e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:34.952544 21149 tablet_bootstrap.cc:492] T 1657de8c2c184559b684c9ad9e23346e P ad6563f6630440b2afc6ba67e966bebf: Bootstrap starting.
I20260809 08:52:34.952912 21149 tablet_bootstrap.cc:654] T 1657de8c2c184559b684c9ad9e23346e P ad6563f6630440b2afc6ba67e966bebf: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:34.953545 21151 tablet_bootstrap.cc:492] T 1657de8c2c184559b684c9ad9e23346e P 145b09aa53c74c73a021795b1fa0e19c: Bootstrap starting.
I20260809 08:52:34.953711 21155 tablet_bootstrap.cc:492] T 1657de8c2c184559b684c9ad9e23346e P a7984b066bfa4a7cbea4fd04bd72c6d8: Bootstrap starting.
I20260809 08:52:34.953846 21149 tablet_bootstrap.cc:492] T 1657de8c2c184559b684c9ad9e23346e P ad6563f6630440b2afc6ba67e966bebf: No bootstrap required, opened a new log
I20260809 08:52:34.953903 21149 ts_tablet_manager.cc:1397] T 1657de8c2c184559b684c9ad9e23346e P ad6563f6630440b2afc6ba67e966bebf: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:34.954056 21149 raft_consensus.cc:348] T 1657de8c2c184559b684c9ad9e23346e P ad6563f6630440b2afc6ba67e966bebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad6563f6630440b2afc6ba67e966bebf" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 44029 } } peers { permanent_uuid: "a7984b066bfa4a7cbea4fd04bd72c6d8" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 32787 } } peers { permanent_uuid: "145b09aa53c74c73a021795b1fa0e19c" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 44751 } }
I20260809 08:52:34.954115 21149 raft_consensus.cc:374] T 1657de8c2c184559b684c9ad9e23346e P ad6563f6630440b2afc6ba67e966bebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:34.954129 21149 raft_consensus.cc:729] T 1657de8c2c184559b684c9ad9e23346e P ad6563f6630440b2afc6ba67e966bebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad6563f6630440b2afc6ba67e966bebf, State: Initialized, Role: FOLLOWER
I20260809 08:52:34.954099 21155 tablet_bootstrap.cc:654] T 1657de8c2c184559b684c9ad9e23346e P a7984b066bfa4a7cbea4fd04bd72c6d8: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:34.954191 21149 consensus_queue.cc:260] T 1657de8c2c184559b684c9ad9e23346e P ad6563f6630440b2afc6ba67e966bebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad6563f6630440b2afc6ba67e966bebf" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 44029 } } peers { permanent_uuid: "a7984b066bfa4a7cbea4fd04bd72c6d8" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 32787 } } peers { permanent_uuid: "145b09aa53c74c73a021795b1fa0e19c" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 44751 } }
I20260809 08:52:34.954348 21149 ts_tablet_manager.cc:1428] T 1657de8c2c184559b684c9ad9e23346e P ad6563f6630440b2afc6ba67e966bebf: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 08:52:34.954953 21155 tablet_bootstrap.cc:492] T 1657de8c2c184559b684c9ad9e23346e P a7984b066bfa4a7cbea4fd04bd72c6d8: No bootstrap required, opened a new log
I20260809 08:52:34.955018 21155 ts_tablet_manager.cc:1397] T 1657de8c2c184559b684c9ad9e23346e P a7984b066bfa4a7cbea4fd04bd72c6d8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:34.954970 21161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:34.954706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45607 (local address 127.8.154.1:48083)
0809 08:52:34.954867 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:34.954925 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:45607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":106,"threads_started":1}
I20260809 08:52:34.955132 21155 raft_consensus.cc:348] T 1657de8c2c184559b684c9ad9e23346e P a7984b066bfa4a7cbea4fd04bd72c6d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad6563f6630440b2afc6ba67e966bebf" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 44029 } } peers { permanent_uuid: "a7984b066bfa4a7cbea4fd04bd72c6d8" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 32787 } } peers { permanent_uuid: "145b09aa53c74c73a021795b1fa0e19c" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 44751 } }
I20260809 08:52:34.955209 21155 raft_consensus.cc:374] T 1657de8c2c184559b684c9ad9e23346e P a7984b066bfa4a7cbea4fd04bd72c6d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:34.955241 21155 raft_consensus.cc:729] T 1657de8c2c184559b684c9ad9e23346e P a7984b066bfa4a7cbea4fd04bd72c6d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7984b066bfa4a7cbea4fd04bd72c6d8, State: Initialized, Role: FOLLOWER
I20260809 08:52:34.955322 21155 consensus_queue.cc:260] T 1657de8c2c184559b684c9ad9e23346e P a7984b066bfa4a7cbea4fd04bd72c6d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad6563f6630440b2afc6ba67e966bebf" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 44029 } } peers { permanent_uuid: "a7984b066bfa4a7cbea4fd04bd72c6d8" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 32787 } } peers { permanent_uuid: "145b09aa53c74c73a021795b1fa0e19c" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 44751 } }
I20260809 08:52:34.955489 21155 ts_tablet_manager.cc:1428] T 1657de8c2c184559b684c9ad9e23346e P a7984b066bfa4a7cbea4fd04bd72c6d8: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 08:52:34.956014 21164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:34.955729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45607 (local address 127.8.154.2:60123)
0809 08:52:34.955908 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:34.955971 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:45607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":80,"threads_started":1}
I20260809 08:52:34.956039 21151 tablet_bootstrap.cc:654] T 1657de8c2c184559b684c9ad9e23346e P 145b09aa53c74c73a021795b1fa0e19c: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:34.956487 20617 heartbeater.cc:503] Master 127.8.154.61:41867 was elected leader, sending a full tablet report...
I20260809 08:52:34.956722 21151 tablet_bootstrap.cc:492] T 1657de8c2c184559b684c9ad9e23346e P 145b09aa53c74c73a021795b1fa0e19c: No bootstrap required, opened a new log
I20260809 08:52:34.956775 21151 ts_tablet_manager.cc:1397] T 1657de8c2c184559b684c9ad9e23346e P 145b09aa53c74c73a021795b1fa0e19c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 08:52:34.956893 21151 raft_consensus.cc:348] T 1657de8c2c184559b684c9ad9e23346e P 145b09aa53c74c73a021795b1fa0e19c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad6563f6630440b2afc6ba67e966bebf" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 44029 } } peers { permanent_uuid: "a7984b066bfa4a7cbea4fd04bd72c6d8" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 32787 } } peers { permanent_uuid: "145b09aa53c74c73a021795b1fa0e19c" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 44751 } }
I20260809 08:52:34.956964 21151 raft_consensus.cc:374] T 1657de8c2c184559b684c9ad9e23346e P 145b09aa53c74c73a021795b1fa0e19c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:34.956985 21151 raft_consensus.cc:729] T 1657de8c2c184559b684c9ad9e23346e P 145b09aa53c74c73a021795b1fa0e19c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145b09aa53c74c73a021795b1fa0e19c, State: Initialized, Role: FOLLOWER
I20260809 08:52:34.957065 21151 consensus_queue.cc:260] T 1657de8c2c184559b684c9ad9e23346e P 145b09aa53c74c73a021795b1fa0e19c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad6563f6630440b2afc6ba67e966bebf" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 44029 } } peers { permanent_uuid: "a7984b066bfa4a7cbea4fd04bd72c6d8" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 32787 } } peers { permanent_uuid: "145b09aa53c74c73a021795b1fa0e19c" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 44751 } }
I20260809 08:52:34.957105 20834 heartbeater.cc:503] Master 127.8.154.61:41867 was elected leader, sending a full tablet report...
I20260809 08:52:34.957258 21151 ts_tablet_manager.cc:1428] T 1657de8c2c184559b684c9ad9e23346e P 145b09aa53c74c73a021795b1fa0e19c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 08:52:34.957301 21164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:34.957153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45607 (local address 127.8.154.2:35673)
0809 08:52:34.957192 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:34.957268 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:45607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260809 08:52:34.957412 21161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:34.956592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45607 (local address 127.8.154.1:56107)
0809 08:52:34.957345 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:34.957379 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:45607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":733}
I20260809 08:52:34.957911 21173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:34.957651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45607 (local address 127.8.154.3:51791)
0809 08:52:34.957831 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:34.957874 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:45607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":113,"threads_started":1}
I20260809 08:52:34.958051 21027 heartbeater.cc:503] Master 127.8.154.61:41867 was elected leader, sending a full tablet report...
I20260809 08:52:34.958333 21173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:34.958221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45607 (local address 127.8.154.3:41795)
0809 08:52:34.958280 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:34.958311 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:45607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260809 08:52:35.148677 21162 raft_consensus.cc:482] T 1657de8c2c184559b684c9ad9e23346e P a7984b066bfa4a7cbea4fd04bd72c6d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:52:35.148800 21162 raft_consensus.cc:504] T 1657de8c2c184559b684c9ad9e23346e P a7984b066bfa4a7cbea4fd04bd72c6d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad6563f6630440b2afc6ba67e966bebf" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 44029 } } peers { permanent_uuid: "a7984b066bfa4a7cbea4fd04bd72c6d8" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 32787 } } peers { permanent_uuid: "145b09aa53c74c73a021795b1fa0e19c" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 44751 } }
I20260809 08:52:35.149012 21162 leader_election.cc:288] T 1657de8c2c184559b684c9ad9e23346e P a7984b066bfa4a7cbea4fd04bd72c6d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad6563f6630440b2afc6ba67e966bebf (127.8.154.1:44029), 145b09aa53c74c73a021795b1fa0e19c (127.8.154.3:44751)
I20260809 08:52:35.149396 20956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1657de8c2c184559b684c9ad9e23346e" candidate_uuid: "a7984b066bfa4a7cbea4fd04bd72c6d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145b09aa53c74c73a021795b1fa0e19c" is_pre_election: true
I20260809 08:52:35.149425 20535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1657de8c2c184559b684c9ad9e23346e" candidate_uuid: "a7984b066bfa4a7cbea4fd04bd72c6d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad6563f6630440b2afc6ba67e966bebf" is_pre_election: true
I20260809 08:52:35.149526 20535 raft_consensus.cc:2454] T 1657de8c2c184559b684c9ad9e23346e P ad6563f6630440b2afc6ba67e966bebf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7984b066bfa4a7cbea4fd04bd72c6d8 in term 0.
I20260809 08:52:35.149528 20956 raft_consensus.cc:2454] T 1657de8c2c184559b684c9ad9e23346e P 145b09aa53c74c73a021795b1fa0e19c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7984b066bfa4a7cbea4fd04bd72c6d8 in term 0.
I20260809 08:52:35.149693 20653 leader_election.cc:302] T 1657de8c2c184559b684c9ad9e23346e P a7984b066bfa4a7cbea4fd04bd72c6d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145b09aa53c74c73a021795b1fa0e19c, a7984b066bfa4a7cbea4fd04bd72c6d8; no voters:
I20260809 08:52:35.149817 21162 raft_consensus.cc:2781] T 1657de8c2c184559b684c9ad9e23346e P a7984b066bfa4a7cbea4fd04bd72c6d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:52:35.149906 21162 raft_consensus.cc:482] T 1657de8c2c184559b684c9ad9e23346e P a7984b066bfa4a7cbea4fd04bd72c6d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:52:35.149938 21162 raft_consensus.cc:3037] T 1657de8c2c184559b684c9ad9e23346e P a7984b066bfa4a7cbea4fd04bd72c6d8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:35.150691 21162 raft_consensus.cc:504] T 1657de8c2c184559b684c9ad9e23346e P a7984b066bfa4a7cbea4fd04bd72c6d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad6563f6630440b2afc6ba67e966bebf" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 44029 } } peers { permanent_uuid: "a7984b066bfa4a7cbea4fd04bd72c6d8" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 32787 } } peers { permanent_uuid: "145b09aa53c74c73a021795b1fa0e19c" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 44751 } }
I20260809 08:52:35.150846 21162 leader_election.cc:288] T 1657de8c2c184559b684c9ad9e23346e P a7984b066bfa4a7cbea4fd04bd72c6d8 [CANDIDATE]: Term 1 election: Requested vote from peers ad6563f6630440b2afc6ba67e966bebf (127.8.154.1:44029), 145b09aa53c74c73a021795b1fa0e19c (127.8.154.3:44751)
I20260809 08:52:35.151036 20956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1657de8c2c184559b684c9ad9e23346e" candidate_uuid: "a7984b066bfa4a7cbea4fd04bd72c6d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "145b09aa53c74c73a021795b1fa0e19c"
I20260809 08:52:35.151036 20535 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1657de8c2c184559b684c9ad9e23346e" candidate_uuid: "a7984b066bfa4a7cbea4fd04bd72c6d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad6563f6630440b2afc6ba67e966bebf"
I20260809 08:52:35.151124 20535 raft_consensus.cc:3037] T 1657de8c2c184559b684c9ad9e23346e P ad6563f6630440b2afc6ba67e966bebf [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:35.151124 20956 raft_consensus.cc:3037] T 1657de8c2c184559b684c9ad9e23346e P 145b09aa53c74c73a021795b1fa0e19c [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:35.151962 20535 raft_consensus.cc:2454] T 1657de8c2c184559b684c9ad9e23346e P ad6563f6630440b2afc6ba67e966bebf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7984b066bfa4a7cbea4fd04bd72c6d8 in term 1.
I20260809 08:52:35.151961 20956 raft_consensus.cc:2454] T 1657de8c2c184559b684c9ad9e23346e P 145b09aa53c74c73a021795b1fa0e19c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7984b066bfa4a7cbea4fd04bd72c6d8 in term 1.
I20260809 08:52:35.152218 20653 leader_election.cc:302] T 1657de8c2c184559b684c9ad9e23346e P a7984b066bfa4a7cbea4fd04bd72c6d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 145b09aa53c74c73a021795b1fa0e19c, a7984b066bfa4a7cbea4fd04bd72c6d8; no voters:
I20260809 08:52:35.152318 21162 raft_consensus.cc:2781] T 1657de8c2c184559b684c9ad9e23346e P a7984b066bfa4a7cbea4fd04bd72c6d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:52:35.152485 21162 raft_consensus.cc:686] T 1657de8c2c184559b684c9ad9e23346e P a7984b066bfa4a7cbea4fd04bd72c6d8 [term 1 LEADER]: Becoming Leader. State: Replica: a7984b066bfa4a7cbea4fd04bd72c6d8, State: Running, Role: LEADER
I20260809 08:52:35.152530 21162 consensus_queue.cc:237] T 1657de8c2c184559b684c9ad9e23346e P a7984b066bfa4a7cbea4fd04bd72c6d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad6563f6630440b2afc6ba67e966bebf" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 44029 } } peers { permanent_uuid: "a7984b066bfa4a7cbea4fd04bd72c6d8" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 32787 } } peers { permanent_uuid: "145b09aa53c74c73a021795b1fa0e19c" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 44751 } }
I20260809 08:52:35.152912 21164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:35.152741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45607 (local address 127.8.154.2:56961)
0809 08:52:35.152821 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:35.152880 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:45607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 08:52:35.153457 20034 catalog_manager.cc:5495] T 1657de8c2c184559b684c9ad9e23346e P a7984b066bfa4a7cbea4fd04bd72c6d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a7984b066bfa4a7cbea4fd04bd72c6d8 (127.8.154.2). New cstate: current_term: 1 leader_uuid: "a7984b066bfa4a7cbea4fd04bd72c6d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad6563f6630440b2afc6ba67e966bebf" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 44029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a7984b066bfa4a7cbea4fd04bd72c6d8" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 32787 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "145b09aa53c74c73a021795b1fa0e19c" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 44751 } health_report { overall_health: UNKNOWN } } }
I20260809 08:52:35.177661 21147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:35.175313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55386 (local address 127.8.154.2:32787)
0809 08:52:35.175380 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:35.175382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:35.175391 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:52:35.175439 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:35.175440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:35.175461 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:35.175528 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:52:35.175676 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:35.176187 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:35.176950 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:35.177151 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:35.177231 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:35.177375 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:52:35.177414 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:52:35.177477 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:35.177538 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 08:52:35.177570 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 08:52:35.178691 20956 raft_consensus.cc:1261] T 1657de8c2c184559b684c9ad9e23346e P 145b09aa53c74c73a021795b1fa0e19c [term 1 FOLLOWER]: Refusing update from remote peer a7984b066bfa4a7cbea4fd04bd72c6d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:52:35.178777 20535 raft_consensus.cc:1261] T 1657de8c2c184559b684c9ad9e23346e P ad6563f6630440b2afc6ba67e966bebf [term 1 FOLLOWER]: Refusing update from remote peer a7984b066bfa4a7cbea4fd04bd72c6d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:52:35.178952 21174 consensus_queue.cc:1035] T 1657de8c2c184559b684c9ad9e23346e P a7984b066bfa4a7cbea4fd04bd72c6d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "145b09aa53c74c73a021795b1fa0e19c" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 44751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:52:35.179140 21162 consensus_queue.cc:1035] T 1657de8c2c184559b684c9ad9e23346e P a7984b066bfa4a7cbea4fd04bd72c6d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad6563f6630440b2afc6ba67e966bebf" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 44029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:52:35.180176 21161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:35.179993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45607 (local address 127.8.154.1:46875)
0809 08:52:35.180100 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:35.180138 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:45607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260809 08:52:35.180334 21173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:35.180001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45607 (local address 127.8.154.3:49117)
0809 08:52:35.180273 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:35.180305 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:45607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 08:52:35.181520 21164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:35.181422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45607 (local address 127.8.154.2:51113)
0809 08:52:35.181459 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:35.181489 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:45607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260809 08:52:35.181923 21164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:35.181735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45607 (local address 127.8.154.2:50869)
0809 08:52:35.181861 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:35.181893 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:45607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260809 08:52:35.182170 21173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:35.182048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45607 (local address 127.8.154.3:60723)
0809 08:52:35.182111 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:35.182139 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:45607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 08:52:35.182499 21161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:35.182373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45607 (local address 127.8.154.1:59825)
0809 08:52:35.182443 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:35.182471 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:45607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 08:52:35.184103 20035 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:52:35.184203 20035 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41146:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:52:35.185786 20035 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a9f86b88da034484ab3f19ad3b945ade: Sending DeleteTablet for 3 replicas of tablet 1657de8c2c184559b684c9ad9e23346e
I20260809 08:52:35.186146 20504 tablet_service.cc:1514] Processing DeleteTablet for tablet 1657de8c2c184559b684c9ad9e23346e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:35 UTC) from {username='oryx', principal='oryx/127.8.154.61@KRBTEST.COM'} at 127.0.0.1:59338
I20260809 08:52:35.186162 20930 tablet_service.cc:1514] Processing DeleteTablet for tablet 1657de8c2c184559b684c9ad9e23346e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:35 UTC) from {username='oryx', principal='oryx/127.8.154.61@KRBTEST.COM'} at 127.0.0.1:42386
I20260809 08:52:35.186357 21195 tablet_replica.cc:330] T 1657de8c2c184559b684c9ad9e23346e P ad6563f6630440b2afc6ba67e966bebf: stopping tablet replica
I20260809 08:52:35.186389 21196 tablet_replica.cc:330] T 1657de8c2c184559b684c9ad9e23346e P 145b09aa53c74c73a021795b1fa0e19c: stopping tablet replica
I20260809 08:52:35.186424 20720 tablet_service.cc:1514] Processing DeleteTablet for tablet 1657de8c2c184559b684c9ad9e23346e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:35 UTC) from {username='oryx', principal='oryx/127.8.154.61@KRBTEST.COM'} at 127.0.0.1:55372
I20260809 08:52:35.186456 21195 raft_consensus.cc:2229] T 1657de8c2c184559b684c9ad9e23346e P ad6563f6630440b2afc6ba67e966bebf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:52:35.186568 21195 raft_consensus.cc:2258] T 1657de8c2c184559b684c9ad9e23346e P ad6563f6630440b2afc6ba67e966bebf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:52:35.186527 21196 raft_consensus.cc:2229] T 1657de8c2c184559b684c9ad9e23346e P 145b09aa53c74c73a021795b1fa0e19c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:52:35.186615 21197 tablet_replica.cc:330] T 1657de8c2c184559b684c9ad9e23346e P a7984b066bfa4a7cbea4fd04bd72c6d8: stopping tablet replica
I20260809 08:52:35.186663 21196 raft_consensus.cc:2258] T 1657de8c2c184559b684c9ad9e23346e P 145b09aa53c74c73a021795b1fa0e19c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:52:35.186692 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 20271
I20260809 08:52:35.187177 21197 raft_consensus.cc:2229] T 1657de8c2c184559b684c9ad9e23346e P a7984b066bfa4a7cbea4fd04bd72c6d8 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:52:35.187265 21197 raft_consensus.cc:2258] T 1657de8c2c184559b684c9ad9e23346e P a7984b066bfa4a7cbea4fd04bd72c6d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:52:35.187660 21197 ts_tablet_manager.cc:1905] T 1657de8c2c184559b684c9ad9e23346e P a7984b066bfa4a7cbea4fd04bd72c6d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:52:35.189353 21197 ts_tablet_manager.cc:1918] T 1657de8c2c184559b684c9ad9e23346e P a7984b066bfa4a7cbea4fd04bd72c6d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:52:35.189431 21197 log.cc:1196] T 1657de8c2c184559b684c9ad9e23346e P a7984b066bfa4a7cbea4fd04bd72c6d8: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-1/wal/wals/1657de8c2c184559b684c9ad9e23346e
I20260809 08:52:35.189692 21197 ts_tablet_manager.cc:1939] T 1657de8c2c184559b684c9ad9e23346e P a7984b066bfa4a7cbea4fd04bd72c6d8: Deleting consensus metadata
I20260809 08:52:35.190034 20015 catalog_manager.cc:4841] TS a7984b066bfa4a7cbea4fd04bd72c6d8 (127.8.154.2:32787): tablet 1657de8c2c184559b684c9ad9e23346e (table test-table [id=82f8cab4c5e64d57afcdb69ed1586d71]) successfully deleted
I20260809 08:52:35.190369 21196 ts_tablet_manager.cc:1905] T 1657de8c2c184559b684c9ad9e23346e P 145b09aa53c74c73a021795b1fa0e19c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:52:35.192178 21196 ts_tablet_manager.cc:1918] T 1657de8c2c184559b684c9ad9e23346e P 145b09aa53c74c73a021795b1fa0e19c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:52:35.192256 21196 log.cc:1196] T 1657de8c2c184559b684c9ad9e23346e P 145b09aa53c74c73a021795b1fa0e19c: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265519942966-8808-0/minicluster-data/ts-2/wal/wals/1657de8c2c184559b684c9ad9e23346e
I20260809 08:52:35.192481 21196 ts_tablet_manager.cc:1939] T 1657de8c2c184559b684c9ad9e23346e P 145b09aa53c74c73a021795b1fa0e19c: Deleting consensus metadata
I20260809 08:52:35.192838 20017 catalog_manager.cc:4841] TS 145b09aa53c74c73a021795b1fa0e19c (127.8.154.3:44751): tablet 1657de8c2c184559b684c9ad9e23346e (table test-table [id=82f8cab4c5e64d57afcdb69ed1586d71]) successfully deleted
I20260809 08:52:35.202286 21146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:35.202125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:44029 (local address 127.0.0.1:59354)
0809 08:52:35.202198 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:35.202240 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.1:44029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 08:52:35.202275 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 20625
W20260809 08:52:35.208039 20017 catalog_manager.cc:4584] TS ad6563f6630440b2afc6ba67e966bebf (127.8.154.1:44029): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1657de8c2c184559b684c9ad9e23346e: Network error: Client connection negotiation failed: client connection to 127.8.154.1:44029: connect: Connection refused (error 111)
I20260809 08:52:35.213902 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 20839
I20260809 08:52:35.227028 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 19818
I20260809 08:52:35.239428 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 20122
2026-08-09T08:52:35Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5356 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:52:35 dist-test-slave-t3q3 krb5kdc[21203](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:52:35 dist-test-slave-t3q3 krb5kdc[21203](info): set up 2 sockets
Aug 09 08:52:35 dist-test-slave-t3q3 krb5kdc[21203](info): commencing operation
krb5kdc: starting...
W20260809 08:52:36.509035 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:52:36 dist-test-slave-t3q3 krb5kdc[21203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:52:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:52:36Z Disabled control of system clock
WARNING: no policy specified for oryx/127.8.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.62.keytab.
Entry for principal oryx/127.8.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.62.keytab.
I20260809 08:52:36.631295 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:42005
--webserver_interface=127.8.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.62.keytab
--principal=oryx/127.8.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:44675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:42005
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:36.715266 21225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:36.715497 21225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:36.715521 21225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:36.716847 21225 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:52:36.716897 21225 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 08:52:36.716912 21225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:36.716926 21225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:36.716990 21225 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:52:36.717031 21225 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:52:36.718726 21225 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:44675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:42005
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.62.keytab
--principal=oryx/127.8.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:42005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:36.719024 21225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:36.719286 21225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:36.724529 21243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:36.724615 21237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:36.724665 21225 server_base.cc:1034] running on GCE node
W20260809 08:52:36.724634 21234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:36.725117 21225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:36.725446 21225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:36.726621 21225 hybrid_clock.cc:648] HybridClock initialized: now 1786265556726619 us; error 38 us; skew 500 ppm
Aug 09 08:52:36 dist-test-slave-t3q3 krb5kdc[21203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265556, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:36.730059 21225 init.cc:377] Logged in from keytab as oryx/127.8.154.62@KRBTEST.COM (short username oryx)
I20260809 08:52:36.730669 21225 webserver.cc:460] Webserver started at http://127.8.154.62:33365/ using document root <none> and password file <none>
I20260809 08:52:36.730984 21225 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:36.731072 21225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:36.731216 21225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:36.732920 21225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "eea4f50c735c4e06b19ef831930e79b9"
format_stamp: "Formatted at 2026-08-09 08:52:36 on dist-test-slave-t3q3"
server_key: "54728640a2312da0f4513f9c85acc447"
server_key_iv: "adea59db9ed6b9d990700de68e3c7c64"
server_key_version: "encryptionkey@0"
I20260809 08:52:36.733420 21225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "eea4f50c735c4e06b19ef831930e79b9"
format_stamp: "Formatted at 2026-08-09 08:52:36 on dist-test-slave-t3q3"
server_key: "54728640a2312da0f4513f9c85acc447"
server_key_iv: "adea59db9ed6b9d990700de68e3c7c64"
server_key_version: "encryptionkey@0"
I20260809 08:52:36.735450 21225 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 08:52:36.736541 21255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:36.736742 21225 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 08:52:36.736845 21225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "eea4f50c735c4e06b19ef831930e79b9"
format_stamp: "Formatted at 2026-08-09 08:52:36 on dist-test-slave-t3q3"
server_key: "54728640a2312da0f4513f9c85acc447"
server_key_iv: "adea59db9ed6b9d990700de68e3c7c64"
server_key_version: "encryptionkey@0"
I20260809 08:52:36.736948 21225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:36.752143 21225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:36.754079 21225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:36.754192 21225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:36.758852 21225 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:42005
I20260809 08:52:36.758941 21358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:42005 every 8 connection(s)
I20260809 08:52:36.759303 21225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:52:36.760035 21360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:36.762378 21360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eea4f50c735c4e06b19ef831930e79b9: Bootstrap starting.
I20260809 08:52:36.762993 21360 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eea4f50c735c4e06b19ef831930e79b9: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:36.763304 21360 log.cc:824] T 00000000000000000000000000000000 P eea4f50c735c4e06b19ef831930e79b9: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:36.764142 21360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eea4f50c735c4e06b19ef831930e79b9: No bootstrap required, opened a new log
I20260809 08:52:36.766175 21360 raft_consensus.cc:348] T 00000000000000000000000000000000 P eea4f50c735c4e06b19ef831930e79b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eea4f50c735c4e06b19ef831930e79b9" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 42005 } }
I20260809 08:52:36.766340 21360 raft_consensus.cc:374] T 00000000000000000000000000000000 P eea4f50c735c4e06b19ef831930e79b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:36.766391 21360 raft_consensus.cc:729] T 00000000000000000000000000000000 P eea4f50c735c4e06b19ef831930e79b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eea4f50c735c4e06b19ef831930e79b9, State: Initialized, Role: FOLLOWER
I20260809 08:52:36.766439 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 21225
I20260809 08:52:36.766546 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:52:36.766575 21360 consensus_queue.cc:260] T 00000000000000000000000000000000 P eea4f50c735c4e06b19ef831930e79b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eea4f50c735c4e06b19ef831930e79b9" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 42005 } }
I20260809 08:52:36.766673 21360 raft_consensus.cc:388] T 00000000000000000000000000000000 P eea4f50c735c4e06b19ef831930e79b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:52:36.766717 21360 raft_consensus.cc:482] T 00000000000000000000000000000000 P eea4f50c735c4e06b19ef831930e79b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:52:36.766770 21360 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eea4f50c735c4e06b19ef831930e79b9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:36.766817 8808 external_mini_cluster.cc:1428] Setting key 7e58ac6a881b078ade7b15b6af86ee6d
I20260809 08:52:36.767443 21360 raft_consensus.cc:504] T 00000000000000000000000000000000 P eea4f50c735c4e06b19ef831930e79b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eea4f50c735c4e06b19ef831930e79b9" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 42005 } }
I20260809 08:52:36.767580 21360 leader_election.cc:302] T 00000000000000000000000000000000 P eea4f50c735c4e06b19ef831930e79b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eea4f50c735c4e06b19ef831930e79b9; no voters:
I20260809 08:52:36.767763 21360 leader_election.cc:288] T 00000000000000000000000000000000 P eea4f50c735c4e06b19ef831930e79b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:52:36.767794 21371 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eea4f50c735c4e06b19ef831930e79b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:52:36.767921 21371 raft_consensus.cc:686] T 00000000000000000000000000000000 P eea4f50c735c4e06b19ef831930e79b9 [term 1 LEADER]: Becoming Leader. State: Replica: eea4f50c735c4e06b19ef831930e79b9, State: Running, Role: LEADER
I20260809 08:52:36.768080 21360 sys_catalog.cc:564] T 00000000000000000000000000000000 P eea4f50c735c4e06b19ef831930e79b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:52:36.768100 21371 consensus_queue.cc:237] T 00000000000000000000000000000000 P eea4f50c735c4e06b19ef831930e79b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eea4f50c735c4e06b19ef831930e79b9" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 42005 } }
I20260809 08:52:36.768734 21372 sys_catalog.cc:455] T 00000000000000000000000000000000 P eea4f50c735c4e06b19ef831930e79b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eea4f50c735c4e06b19ef831930e79b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eea4f50c735c4e06b19ef831930e79b9" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 42005 } } }
I20260809 08:52:36.768826 21372 sys_catalog.cc:458] T 00000000000000000000000000000000 P eea4f50c735c4e06b19ef831930e79b9 [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:36.768997 21373 sys_catalog.cc:455] T 00000000000000000000000000000000 P eea4f50c735c4e06b19ef831930e79b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eea4f50c735c4e06b19ef831930e79b9. Latest consensus state: current_term: 1 leader_uuid: "eea4f50c735c4e06b19ef831930e79b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eea4f50c735c4e06b19ef831930e79b9" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 42005 } } }
I20260809 08:52:36.769045 21373 sys_catalog.cc:458] T 00000000000000000000000000000000 P eea4f50c735c4e06b19ef831930e79b9 [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:36.769054 21381 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:52:36.769608 21381 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:52:36.771354 21381 catalog_manager.cc:1347] Generated new cluster ID: b8bfd618631a42ac96ba91adf53d0f6c
I20260809 08:52:36.771425 21381 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 08:52:36 dist-test-slave-t3q3 krb5kdc[21203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM
I20260809 08:52:36.779282 21369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:36.767384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38268 (local address 127.8.154.62:42005)
0809 08:52:36.767559 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:36.767569 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:36.767604 (+ 35us) server_negotiation.cc:407] Connection header received
0809 08:52:36.767749 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:36.767755 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:36.767799 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:36.768705 (+ 906us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:36.768828 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:36.770324 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:36.771311 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:36.771966 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:36.772108 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:36.774844 (+ 2736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:36.774848 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:36.774849 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:36.774861 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:36.777537 (+ 2676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:36.777964 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:36.777968 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:36.777968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:36.778092 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:36.778308 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:36.778309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:36.778309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:36.778650 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:36.778829 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:36.778902 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 08:52:36.779050 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":74,"threads_started":1}
I20260809 08:52:36.790030 21381 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:52:36.790557 21381 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:52:36.793488 21381 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eea4f50c735c4e06b19ef831930e79b9: Generated new TSK 0
I20260809 08:52:36.793684 21381 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.8.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.1.keytab.
Entry for principal oryx/127.8.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.1.keytab.
I20260809 08:52:36.827129 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.1:0
--local_ip_for_outbound_sockets=127.8.154.1
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:42005
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.1.keytab
--principal=oryx/127.8.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:44675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:36.911298 21401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:36.911507 21401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:36.911531 21401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:36.912822 21401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:36.912868 21401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:36.912889 21401 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 08:52:36.912907 21401 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.1
I20260809 08:52:36.914448 21401 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:44675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.1.keytab
--principal=oryx/127.8.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.154.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.154.62:42005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.1
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:36.914705 21401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:36.914947 21401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:36.920241 21406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:36.920378 21401 server_base.cc:1034] running on GCE node
W20260809 08:52:36.920257 21412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:36.920630 21409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:36.920889 21401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:36.921099 21401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:36.922252 21401 hybrid_clock.cc:648] HybridClock initialized: now 1786265556922239 us; error 26 us; skew 500 ppm
Aug 09 08:52:36 dist-test-slave-t3q3 krb5kdc[21203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265556, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:36.924829 21401 init.cc:377] Logged in from keytab as oryx/127.8.154.1@KRBTEST.COM (short username oryx)
I20260809 08:52:36.925292 21401 webserver.cc:460] Webserver started at http://127.8.154.1:35257/ using document root <none> and password file <none>
I20260809 08:52:36.925474 21401 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:36.925514 21401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:36.925602 21401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:36.926608 21401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-0/data/instance:
uuid: "1b449493e13e45b78f81076c14afe878"
format_stamp: "Formatted at 2026-08-09 08:52:36 on dist-test-slave-t3q3"
server_key: "e50daedf278b8db358d283d31ecb96f9"
server_key_iv: "77ff949bd1eed26d81c1330f288185d6"
server_key_version: "encryptionkey@0"
I20260809 08:52:36.926908 21401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance:
uuid: "1b449493e13e45b78f81076c14afe878"
format_stamp: "Formatted at 2026-08-09 08:52:36 on dist-test-slave-t3q3"
server_key: "e50daedf278b8db358d283d31ecb96f9"
server_key_iv: "77ff949bd1eed26d81c1330f288185d6"
server_key_version: "encryptionkey@0"
I20260809 08:52:36.928285 21401 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 08:52:36.928972 21421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:36.929147 21401 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:52:36.929260 21401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-0/wal
uuid: "1b449493e13e45b78f81076c14afe878"
format_stamp: "Formatted at 2026-08-09 08:52:36 on dist-test-slave-t3q3"
server_key: "e50daedf278b8db358d283d31ecb96f9"
server_key_iv: "77ff949bd1eed26d81c1330f288185d6"
server_key_version: "encryptionkey@0"
I20260809 08:52:36.929338 21401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:36.938758 21401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:36.941293 21401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:36.941471 21401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:36.942049 21401 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:36.942121 21401 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:36.942171 21401 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:36.942202 21401 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:36.949051 21401 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.1:40933
I20260809 08:52:36.949144 21603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.1:40933 every 8 connection(s)
Aug 09 08:52:36 dist-test-slave-t3q3 krb5kdc[21203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265556, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM
I20260809 08:52:36.949555 21401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
I20260809 08:52:36.952925 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 21401
I20260809 08:52:36.953065 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance
I20260809 08:52:36.953074 21369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:36.942454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:40973 (local address 127.8.154.62:42005)
0809 08:52:36.942509 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:36.942512 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:36.942726 (+ 214us) server_negotiation.cc:407] Connection header received
0809 08:52:36.943291 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:36.943293 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:36.943307 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:36.943362 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:36.944228 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:36.945055 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:36.946620 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:36.946968 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:36.947034 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:36.950053 (+ 3019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:36.950055 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:36.950056 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:36.950067 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:36.951531 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:36.951973 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:36.951974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:36.951974 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:36.952029 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:36.952359 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:36.952359 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:36.952360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:36.952472 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:36.952527 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:36.952882 (+ 355us) server_negotiation.cc:299] Negotiation successful
0809 08:52:36.952951 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 08:52:36.953292 8808 external_mini_cluster.cc:1428] Setting key cf2784f50da1a79972f8a9f934e1bcd3
I20260809 08:52:36.953321 21437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:36.942438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:42005 (local address 127.8.154.1:40973)
0809 08:52:36.942598 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:36.942638 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:36.943140 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:36.943436 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:36.943439 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:36.943784 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:36.944109 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:36.944117 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:36.945145 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:36.945147 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:36.946510 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:36.946513 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:36.947106 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:36.947107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:36.947221 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:36.947812 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:36.947826 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:36.949937 (+ 2111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:36.951631 (+ 1694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:36.951639 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:36.951641 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:36.951883 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:36.952180 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:36.952181 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:36.952182 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:36.952296 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:36.952554 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:36.952562 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:36.952724 (+ 162us) client_negotiation.cc:769] Sending connection context
0809 08:52:36.952853 (+ 129us) client_negotiation.cc:240] Negotiation successful
0809 08:52:36.953078 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":114,"threads_started":1}
I20260809 08:52:36.953912 21604 heartbeater.cc:347] Connected to a master server at 127.8.154.62:42005
I20260809 08:52:36.954061 21604 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:36.954275 21604 heartbeater.cc:511] Master 127.8.154.62:42005 requested a full tablet report, sending...
I20260809 08:52:36.954852 21282 ts_manager.cc:194] Registered new tserver with Master: 1b449493e13e45b78f81076c14afe878 (127.8.154.1:40933)
I20260809 08:52:36.955753 21282 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.154.1@KRBTEST.COM'} at 127.8.154.1:40973
I20260809 08:52:36.960850 21369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:36.955117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:41497 (local address 127.8.154.62:42005)
0809 08:52:36.955156 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:36.955158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:36.955202 (+ 44us) server_negotiation.cc:407] Connection header received
0809 08:52:36.955275 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:36.955276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:36.955286 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:36.955329 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:36.955676 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:36.956320 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:36.957055 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:36.957265 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:36.957343 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:36.958319 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:36.958321 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:36.958321 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:36.958327 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:36.959679 (+ 1352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:36.960127 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:36.960128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:36.960128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:36.960179 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:36.960396 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:36.960396 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:36.960397 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:36.960491 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:36.960542 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:36.960730 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 08:52:36.960793 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:52:36.961118 21614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:36.954965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:42005 (local address 127.8.154.1:41497)
0809 08:52:36.955144 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:36.955155 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:36.955218 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:36.955343 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:36.955344 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:36.955526 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:36.955586 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:36.955588 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:36.956419 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:36.956420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:36.956952 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:36.956953 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:36.957341 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:36.957342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:36.957415 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:36.957820 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:36.957825 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:36.958231 (+ 406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:36.959766 (+ 1535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:36.959768 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:36.959768 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:36.960018 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:36.960251 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:36.960252 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:36.960253 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:36.960328 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:36.960567 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:36.960568 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:36.960625 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 08:52:36.960715 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 08:52:36.960834 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":72,"threads_started":1}
WARNING: no policy specified for oryx/127.8.154.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.154.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.2.keytab.
Entry for principal oryx/127.8.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.2.keytab.
I20260809 08:52:36.982317 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.2:0
--local_ip_for_outbound_sockets=127.8.154.2
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:42005
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.2.keytab
--principal=oryx/127.8.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:44675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:37.066114 21617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:37.066315 21617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:37.066339 21617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:37.067982 21617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:37.068029 21617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:37.068050 21617 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 08:52:37.068123 21617 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.2
I20260809 08:52:37.069821 21617 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:44675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.2.keytab
--principal=oryx/127.8.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.154.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.154.62:42005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.2
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:37.070026 21617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:37.070308 21617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:37.075605 21624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:37.075604 21630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:37.075593 21628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:37.075834 21617 server_base.cc:1034] running on GCE node
I20260809 08:52:37.076135 21617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:37.076401 21617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:37.077569 21617 hybrid_clock.cc:648] HybridClock initialized: now 1786265557077554 us; error 37 us; skew 500 ppm
Aug 09 08:52:37 dist-test-slave-t3q3 krb5kdc[21203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265557, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:37.080210 21617 init.cc:377] Logged in from keytab as oryx/127.8.154.2@KRBTEST.COM (short username oryx)
I20260809 08:52:37.080657 21617 webserver.cc:460] Webserver started at http://127.8.154.2:34057/ using document root <none> and password file <none>
I20260809 08:52:37.080914 21617 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:37.080982 21617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:37.081122 21617 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:37.082178 21617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-1/data/instance:
uuid: "0989b9cc02824c4fb1902322ef469ed2"
format_stamp: "Formatted at 2026-08-09 08:52:37 on dist-test-slave-t3q3"
server_key: "d944012e223988a919526bb139927848"
server_key_iv: "ded28ff70b03c28f8fbf94244cac7ddd"
server_key_version: "encryptionkey@0"
I20260809 08:52:37.082621 21617 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance:
uuid: "0989b9cc02824c4fb1902322ef469ed2"
format_stamp: "Formatted at 2026-08-09 08:52:37 on dist-test-slave-t3q3"
server_key: "d944012e223988a919526bb139927848"
server_key_iv: "ded28ff70b03c28f8fbf94244cac7ddd"
server_key_version: "encryptionkey@0"
I20260809 08:52:37.084050 21617 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 08:52:37.084862 21644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:37.085086 21617 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:52:37.085201 21617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-1/wal
uuid: "0989b9cc02824c4fb1902322ef469ed2"
format_stamp: "Formatted at 2026-08-09 08:52:37 on dist-test-slave-t3q3"
server_key: "d944012e223988a919526bb139927848"
server_key_iv: "ded28ff70b03c28f8fbf94244cac7ddd"
server_key_version: "encryptionkey@0"
I20260809 08:52:37.085286 21617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:37.099273 21617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:37.101290 21617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:37.101461 21617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:37.101976 21617 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:37.102051 21617 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:37.102097 21617 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:37.102125 21617 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 08:52:37 dist-test-slave-t3q3 krb5kdc[21203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265557, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM
I20260809 08:52:37.110013 21617 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.2:42803
I20260809 08:52:37.110090 21823 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.2:42803 every 8 connection(s)
I20260809 08:52:37.110395 21617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
I20260809 08:52:37.112524 21369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:37.102520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:58069 (local address 127.8.154.62:42005)
0809 08:52:37.102595 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:37.102597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:37.102800 (+ 203us) server_negotiation.cc:407] Connection header received
0809 08:52:37.103547 (+ 747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:37.103548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:37.103559 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:37.103605 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:37.104598 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.105117 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.106847 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.107051 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.107132 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:37.109418 (+ 2286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:37.109420 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:37.109420 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:37.109429 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:37.111077 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:37.111500 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:37.111501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:37.111502 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:37.111567 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:37.111828 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:37.111829 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:37.111829 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:37.111952 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:37.112026 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:37.112342 (+ 316us) server_negotiation.cc:299] Negotiation successful
0809 08:52:37.112434 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 08:52:37.112694 21658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:37.102446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:42005 (local address 127.8.154.2:58069)
0809 08:52:37.102648 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:37.102699 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:37.103432 (+ 733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:37.103688 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:37.103693 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:37.104160 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:37.104484 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:37.104493 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.105195 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.105196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:37.106723 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:37.106727 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.107159 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.107160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:37.107304 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:37.107845 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:37.107854 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:37.109316 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:37.111175 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:37.111182 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:37.111185 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:37.111409 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:37.111636 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:37.111637 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:37.111638 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:37.111758 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:37.112031 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:37.112039 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:37.112197 (+ 158us) client_negotiation.cc:769] Sending connection context
0809 08:52:37.112304 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 08:52:37.112464 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":80,"threads_started":1}
I20260809 08:52:37.113195 21824 heartbeater.cc:347] Connected to a master server at 127.8.154.62:42005
I20260809 08:52:37.113302 21824 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:37.113482 21824 heartbeater.cc:511] Master 127.8.154.62:42005 requested a full tablet report, sending...
I20260809 08:52:37.113890 21279 ts_manager.cc:194] Registered new tserver with Master: 0989b9cc02824c4fb1902322ef469ed2 (127.8.154.2:42803)
I20260809 08:52:37.114428 21279 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.154.2@KRBTEST.COM'} at 127.8.154.2:58069
I20260809 08:52:37.117726 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 21617
I20260809 08:52:37.117852 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance
I20260809 08:52:37.118099 8808 external_mini_cluster.cc:1428] Setting key f36e2b040813a2833378419b13b85262
I20260809 08:52:37.120150 21369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:37.114318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:54835 (local address 127.8.154.62:42005)
0809 08:52:37.114353 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:37.114355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:37.114488 (+ 133us) server_negotiation.cc:407] Connection header received
0809 08:52:37.114583 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:37.114584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:37.114594 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:37.114652 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:37.114993 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.115459 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.116184 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.116378 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.116454 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:37.117384 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:37.117385 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:37.117386 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:37.117392 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:37.118891 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:37.119330 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:37.119331 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:37.119332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:37.119387 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:37.119606 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:37.119607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:37.119607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:37.119751 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:37.119804 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:37.119973 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 08:52:37.120034 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260809 08:52:37.120324 21832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:37.114251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:42005 (local address 127.8.154.2:54835)
0809 08:52:37.114420 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:37.114429 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:37.114504 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:37.114658 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:37.114659 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:37.114850 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:37.114905 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:37.114906 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.115551 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.115552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:37.116115 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:37.116117 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.116454 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.116454 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:37.116527 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:37.116970 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:37.116976 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:37.117275 (+ 299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:37.118980 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:37.118981 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:37.118981 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:37.119250 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:37.119456 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:37.119456 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:37.119457 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:37.119542 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:37.119817 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:37.119818 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:37.119873 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 08:52:37.119947 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 08:52:37.120105 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":74,"threads_started":1}
WARNING: no policy specified for oryx/127.8.154.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.154.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.3.keytab.
Entry for principal oryx/127.8.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.3.keytab.
I20260809 08:52:37.147436 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.3:0
--local_ip_for_outbound_sockets=127.8.154.3
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:42005
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.3.keytab
--principal=oryx/127.8.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:44675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:37.230012 21838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:37.230263 21838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:37.230307 21838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:37.231436 21838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:37.231508 21838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:37.231545 21838 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 08:52:37.231601 21838 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.3
I20260809 08:52:37.233283 21838 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:44675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.3.keytab
--principal=oryx/127.8.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.154.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.8.154.62:42005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.3
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:37.233552 21838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:37.233789 21838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:37.238986 21849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:37.239100 21848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:37.238998 21854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:37.239272 21838 server_base.cc:1034] running on GCE node
I20260809 08:52:37.239526 21838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:37.239768 21838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:37.240911 21838 hybrid_clock.cc:648] HybridClock initialized: now 1786265557240899 us; error 29 us; skew 500 ppm
Aug 09 08:52:37 dist-test-slave-t3q3 krb5kdc[21203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265557, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:37.243404 21838 init.cc:377] Logged in from keytab as oryx/127.8.154.3@KRBTEST.COM (short username oryx)
I20260809 08:52:37.243888 21838 webserver.cc:460] Webserver started at http://127.8.154.3:40571/ using document root <none> and password file <none>
I20260809 08:52:37.244171 21838 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:37.244249 21838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:37.244390 21838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:37.245409 21838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-2/data/instance:
uuid: "d8f2b374859b449da01967e60ecea5aa"
format_stamp: "Formatted at 2026-08-09 08:52:37 on dist-test-slave-t3q3"
server_key: "77c6e5ff1f771a89d6d6cdc26e851960"
server_key_iv: "5d2c401536883963e2c45543a7273c51"
server_key_version: "encryptionkey@0"
I20260809 08:52:37.245787 21838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance:
uuid: "d8f2b374859b449da01967e60ecea5aa"
format_stamp: "Formatted at 2026-08-09 08:52:37 on dist-test-slave-t3q3"
server_key: "77c6e5ff1f771a89d6d6cdc26e851960"
server_key_iv: "5d2c401536883963e2c45543a7273c51"
server_key_version: "encryptionkey@0"
I20260809 08:52:37.247282 21838 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:52:37.248150 21865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:37.248353 21838 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:52:37.248477 21838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-2/wal
uuid: "d8f2b374859b449da01967e60ecea5aa"
format_stamp: "Formatted at 2026-08-09 08:52:37 on dist-test-slave-t3q3"
server_key: "77c6e5ff1f771a89d6d6cdc26e851960"
server_key_iv: "5d2c401536883963e2c45543a7273c51"
server_key_version: "encryptionkey@0"
I20260809 08:52:37.248566 21838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:37.264601 21838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:37.266469 21838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:37.266633 21838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:37.267261 21838 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:37.267321 21838 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:37.267359 21838 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:37.267387 21838 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 08:52:37 dist-test-slave-t3q3 krb5kdc[21203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265557, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.3@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM
I20260809 08:52:37.275650 21838 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.3:41705
I20260809 08:52:37.275786 22041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.3:41705 every 8 connection(s)
I20260809 08:52:37.276170 21838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
I20260809 08:52:37.278514 21369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:37.267771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:59983 (local address 127.8.154.62:42005)
0809 08:52:37.267847 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:37.267850 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:37.268090 (+ 240us) server_negotiation.cc:407] Connection header received
0809 08:52:37.268613 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:37.268615 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:37.268628 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:37.268698 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:37.269573 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.270452 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.271988 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.272302 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.272389 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:37.275442 (+ 3053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:37.275444 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:37.275444 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:37.275452 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:37.277313 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:37.277717 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:37.277718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:37.277718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:37.277769 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:37.277969 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:37.277970 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:37.277970 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:37.278089 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:37.278141 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:37.278386 (+ 245us) server_negotiation.cc:299] Negotiation successful
0809 08:52:37.278450 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 08:52:37.278681 21877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:37.267727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:42005 (local address 127.8.154.3:59983)
0809 08:52:37.267928 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:37.267971 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:37.268508 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:37.268799 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:37.268803 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:37.269219 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:37.269471 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:37.269478 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.270572 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.270573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:37.271877 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:37.271880 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.272418 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.272419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:37.272733 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:37.273325 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:37.273335 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:37.275314 (+ 1979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:37.277391 (+ 2077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:37.277397 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:37.277399 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:37.277638 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:37.277815 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:37.277816 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:37.277816 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:37.277915 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:37.278149 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:37.278154 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:37.278266 (+ 112us) client_negotiation.cc:769] Sending connection context
0809 08:52:37.278356 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 08:52:37.278476 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":95,"threads_started":1}
I20260809 08:52:37.279174 22042 heartbeater.cc:347] Connected to a master server at 127.8.154.62:42005
I20260809 08:52:37.279287 22042 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:37.279470 22042 heartbeater.cc:511] Master 127.8.154.62:42005 requested a full tablet report, sending...
I20260809 08:52:37.279893 21279 ts_manager.cc:194] Registered new tserver with Master: d8f2b374859b449da01967e60ecea5aa (127.8.154.3:41705)
I20260809 08:52:37.280467 21279 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.8.154.3@KRBTEST.COM'} at 127.8.154.3:59983
I20260809 08:52:37.283058 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 21838
I20260809 08:52:37.283162 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance
I20260809 08:52:37.283380 8808 external_mini_cluster.cc:1428] Setting key 5deccfd5355d30a3fcfce7e844af334a
I20260809 08:52:37.284509 8808 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:52:37.286345 22050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:37.280296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:42005 (local address 127.8.154.3:40039)
0809 08:52:37.280434 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:37.280445 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:37.280516 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:37.280655 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:37.280656 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:37.280833 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:37.280948 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:37.280950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.281582 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.281583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:37.282105 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:37.282106 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.282480 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.282481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:37.282561 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:37.282947 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:37.282953 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:37.283393 (+ 440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:37.285402 (+ 2009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:37.285403 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:37.285403 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:37.285572 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:37.285780 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:37.285781 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:37.285781 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:37.285829 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:37.286106 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:37.286107 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:37.286143 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 08:52:37.286189 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 08:52:37.286271 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":7,"thread_start_us":86,"threads_started":1}
I20260809 08:52:37.286458 21369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:37.280395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:40039 (local address 127.8.154.62:42005)
0809 08:52:37.280435 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:37.280437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:37.280499 (+ 62us) server_negotiation.cc:407] Connection header received
0809 08:52:37.280579 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:37.280580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:37.280590 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:37.280649 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:37.281021 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.281479 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.282193 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.282402 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.282479 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:37.283489 (+ 1010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:37.283490 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:37.283491 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:37.283497 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:37.285305 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:37.285652 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:37.285653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:37.285654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:37.285720 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:37.285899 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:37.285900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:37.285900 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:37.286045 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:37.286118 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:37.286217 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 08:52:37.286300 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Aug 09 08:52:37 dist-test-slave-t3q3 krb5kdc[21203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265557, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 08:52:37 dist-test-slave-t3q3 krb5kdc[21203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:37 dist-test-slave-t3q3 krb5kdc[21203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
W20260809 08:52:37.307778 21369 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:37.302252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38280 (local address 127.8.154.62:42005)
0809 08:52:37.302321 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:37.302323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:37.302350 (+ 27us) server_negotiation.cc:407] Connection header received
0809 08:52:37.302437 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:37.302438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:37.302446 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:37.302497 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:37.303134 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.303612 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.304532 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.304725 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.304774 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:37.307677 (+ 2903us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38280: BlockingRecv error: recv got EOF from 127.0.0.1:38280 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 09 08:52:37 dist-test-slave-t3q3 krb5kdc[21203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265557, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM
I20260809 08:52:37.315255 21369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:37.308548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38294 (local address 127.8.154.62:42005)
0809 08:52:37.308612 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:37.308613 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:37.308644 (+ 31us) server_negotiation.cc:407] Connection header received
0809 08:52:37.308703 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:37.308703 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:37.308711 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:37.308764 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:37.309186 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.309640 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.310275 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.310446 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.310500 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:37.312650 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:37.312651 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:37.312652 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:37.312660 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:37.314097 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:37.314452 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:37.314453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:37.314454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:37.314515 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:37.314743 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:37.314744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:37.314744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:37.314855 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:37.314963 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:37.315071 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 08:52:37.315152 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 08:52:37.317050 21279 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38294:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:52:37.318554 21279 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:52:37.325731 22090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:37.321397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42604 (local address 127.8.154.3:41705)
0809 08:52:37.321586 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:37.321590 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:37.322041 (+ 451us) server_negotiation.cc:407] Connection header received
0809 08:52:37.322091 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:37.322094 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:37.322111 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:37.322185 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:37.322431 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.323441 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.324734 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.325256 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.325348 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:37.325429 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:37.325580 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 08:52:37.325622 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":91,"threads_started":1}
I20260809 08:52:37.325836 22086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:37.321303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51234 (local address 127.8.154.2:42803)
0809 08:52:37.321500 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:37.321505 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:37.321517 (+ 12us) server_negotiation.cc:407] Connection header received
0809 08:52:37.321559 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:37.321561 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:37.321573 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:37.321658 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:37.321877 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.323026 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.324457 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.325258 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.325336 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:37.325415 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:37.325689 (+ 274us) server_negotiation.cc:299] Negotiation successful
0809 08:52:37.325728 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":91,"threads_started":1}
I20260809 08:52:37.325975 22088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:37.321398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39108 (local address 127.8.154.1:40933)
0809 08:52:37.321661 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:37.321664 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:37.321831 (+ 167us) server_negotiation.cc:407] Connection header received
0809 08:52:37.321956 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:37.321959 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:37.321976 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:37.322071 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:37.322300 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.323025 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.324564 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.325270 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.325335 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:37.325419 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:37.325834 (+ 415us) server_negotiation.cc:299] Negotiation successful
0809 08:52:37.325880 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":93,"threads_started":1}
I20260809 08:52:37.326198 22080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:37.321138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:42803 (local address 127.0.0.1:51234)
0809 08:52:37.321306 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:37.321346 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:37.321418 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:37.321675 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:37.321679 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:37.321681 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:37.321750 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:37.321753 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.323126 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.323127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:37.324368 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:37.324371 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.325456 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.325458 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:37.325532 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:37.325538 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 08:52:37.325590 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 08:52:37.325642 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":46,"thread_start_us":52,"threads_started":1}
I20260809 08:52:37.326231 22081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:37.321152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:41705 (local address 127.0.0.1:42604)
0809 08:52:37.321902 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:37.321930 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:37.321974 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:37.322290 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:37.322291 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:37.322294 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:37.322337 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:37.322339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.323527 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.323528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:37.324636 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:37.324638 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.325350 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.325351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:37.325435 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:37.325451 (+ 16us) client_negotiation.cc:769] Sending connection context
0809 08:52:37.325559 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 08:52:37.325619 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":737,"thread_start_us":104,"threads_started":1}
I20260809 08:52:37.325728 22082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:37.321150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:40933 (local address 127.0.0.1:39108)
0809 08:52:37.321758 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:37.321769 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:37.321822 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:37.322156 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:37.322158 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:37.322160 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:37.322211 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:37.322213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.323119 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.323120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:37.324477 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:37.324481 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.325458 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.325459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:37.325532 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:37.325539 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 08:52:37.325604 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 08:52:37.325642 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"mutex_wait_us":62,"thread_start_us":101,"threads_started":1}
I20260809 08:52:37.327067 21940 tablet_service.cc:1467] Processing CreateTablet for tablet 40d369f545c243c1a3e1d39e88f7aa78 (DEFAULT_TABLE table=test-table [id=81a66db903564b1da15ec7e75a0003a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:37.327099 21496 tablet_service.cc:1467] Processing CreateTablet for tablet 40d369f545c243c1a3e1d39e88f7aa78 (DEFAULT_TABLE table=test-table [id=81a66db903564b1da15ec7e75a0003a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:37.327466 21940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40d369f545c243c1a3e1d39e88f7aa78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:37.327471 21496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40d369f545c243c1a3e1d39e88f7aa78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:37.328691 21722 tablet_service.cc:1467] Processing CreateTablet for tablet 40d369f545c243c1a3e1d39e88f7aa78 (DEFAULT_TABLE table=test-table [id=81a66db903564b1da15ec7e75a0003a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:37.329047 21722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40d369f545c243c1a3e1d39e88f7aa78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:37.330313 22091 tablet_bootstrap.cc:492] T 40d369f545c243c1a3e1d39e88f7aa78 P d8f2b374859b449da01967e60ecea5aa: Bootstrap starting.
I20260809 08:52:37.331177 22091 tablet_bootstrap.cc:654] T 40d369f545c243c1a3e1d39e88f7aa78 P d8f2b374859b449da01967e60ecea5aa: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:37.331563 22091 log.cc:824] T 40d369f545c243c1a3e1d39e88f7aa78 P d8f2b374859b449da01967e60ecea5aa: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:37.331813 22093 tablet_bootstrap.cc:492] T 40d369f545c243c1a3e1d39e88f7aa78 P 1b449493e13e45b78f81076c14afe878: Bootstrap starting.
I20260809 08:52:37.332607 22093 tablet_bootstrap.cc:654] T 40d369f545c243c1a3e1d39e88f7aa78 P 1b449493e13e45b78f81076c14afe878: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:37.332641 22091 tablet_bootstrap.cc:492] T 40d369f545c243c1a3e1d39e88f7aa78 P d8f2b374859b449da01967e60ecea5aa: No bootstrap required, opened a new log
I20260809 08:52:37.332698 22091 ts_tablet_manager.cc:1397] T 40d369f545c243c1a3e1d39e88f7aa78 P d8f2b374859b449da01967e60ecea5aa: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:52:37.332954 22093 log.cc:824] T 40d369f545c243c1a3e1d39e88f7aa78 P 1b449493e13e45b78f81076c14afe878: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:37.333200 22096 tablet_bootstrap.cc:492] T 40d369f545c243c1a3e1d39e88f7aa78 P 0989b9cc02824c4fb1902322ef469ed2: Bootstrap starting.
I20260809 08:52:37.333819 22096 tablet_bootstrap.cc:654] T 40d369f545c243c1a3e1d39e88f7aa78 P 0989b9cc02824c4fb1902322ef469ed2: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:37.333823 22093 tablet_bootstrap.cc:492] T 40d369f545c243c1a3e1d39e88f7aa78 P 1b449493e13e45b78f81076c14afe878: No bootstrap required, opened a new log
I20260809 08:52:37.333907 22093 ts_tablet_manager.cc:1397] T 40d369f545c243c1a3e1d39e88f7aa78 P 1b449493e13e45b78f81076c14afe878: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:52:37.334160 22096 log.cc:824] T 40d369f545c243c1a3e1d39e88f7aa78 P 0989b9cc02824c4fb1902322ef469ed2: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:37.334851 22096 tablet_bootstrap.cc:492] T 40d369f545c243c1a3e1d39e88f7aa78 P 0989b9cc02824c4fb1902322ef469ed2: No bootstrap required, opened a new log
I20260809 08:52:37.334934 22096 ts_tablet_manager.cc:1397] T 40d369f545c243c1a3e1d39e88f7aa78 P 0989b9cc02824c4fb1902322ef469ed2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:52:37.334925 22091 raft_consensus.cc:348] T 40d369f545c243c1a3e1d39e88f7aa78 P d8f2b374859b449da01967e60ecea5aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.335110 22091 raft_consensus.cc:374] T 40d369f545c243c1a3e1d39e88f7aa78 P d8f2b374859b449da01967e60ecea5aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:37.335161 22091 raft_consensus.cc:729] T 40d369f545c243c1a3e1d39e88f7aa78 P d8f2b374859b449da01967e60ecea5aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8f2b374859b449da01967e60ecea5aa, State: Initialized, Role: FOLLOWER
I20260809 08:52:37.335321 22091 consensus_queue.cc:260] T 40d369f545c243c1a3e1d39e88f7aa78 P d8f2b374859b449da01967e60ecea5aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.335632 22042 heartbeater.cc:503] Master 127.8.154.62:42005 was elected leader, sending a full tablet report...
I20260809 08:52:37.335949 22091 ts_tablet_manager.cc:1428] T 40d369f545c243c1a3e1d39e88f7aa78 P d8f2b374859b449da01967e60ecea5aa: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:52:37.335978 22093 raft_consensus.cc:348] T 40d369f545c243c1a3e1d39e88f7aa78 P 1b449493e13e45b78f81076c14afe878 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.336237 22093 raft_consensus.cc:374] T 40d369f545c243c1a3e1d39e88f7aa78 P 1b449493e13e45b78f81076c14afe878 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:37.336282 22093 raft_consensus.cc:729] T 40d369f545c243c1a3e1d39e88f7aa78 P 1b449493e13e45b78f81076c14afe878 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b449493e13e45b78f81076c14afe878, State: Initialized, Role: FOLLOWER
I20260809 08:52:37.336424 22093 consensus_queue.cc:260] T 40d369f545c243c1a3e1d39e88f7aa78 P 1b449493e13e45b78f81076c14afe878 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.336737 22093 ts_tablet_manager.cc:1428] T 40d369f545c243c1a3e1d39e88f7aa78 P 1b449493e13e45b78f81076c14afe878: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:52:37.336751 21604 heartbeater.cc:503] Master 127.8.154.62:42005 was elected leader, sending a full tablet report...
I20260809 08:52:37.337060 22096 raft_consensus.cc:348] T 40d369f545c243c1a3e1d39e88f7aa78 P 0989b9cc02824c4fb1902322ef469ed2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.337224 22096 raft_consensus.cc:374] T 40d369f545c243c1a3e1d39e88f7aa78 P 0989b9cc02824c4fb1902322ef469ed2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:37.337270 22096 raft_consensus.cc:729] T 40d369f545c243c1a3e1d39e88f7aa78 P 0989b9cc02824c4fb1902322ef469ed2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0989b9cc02824c4fb1902322ef469ed2, State: Initialized, Role: FOLLOWER
I20260809 08:52:37.337385 22096 consensus_queue.cc:260] T 40d369f545c243c1a3e1d39e88f7aa78 P 0989b9cc02824c4fb1902322ef469ed2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.337678 22096 ts_tablet_manager.cc:1428] T 40d369f545c243c1a3e1d39e88f7aa78 P 0989b9cc02824c4fb1902322ef469ed2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:52:37.337738 21824 heartbeater.cc:503] Master 127.8.154.62:42005 was elected leader, sending a full tablet report...
W20260809 08:52:37.361048 21825 tablet.cc:2369] T 40d369f545c243c1a3e1d39e88f7aa78 P 0989b9cc02824c4fb1902322ef469ed2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 08:52:37.450425 21606 tablet.cc:2369] T 40d369f545c243c1a3e1d39e88f7aa78 P 1b449493e13e45b78f81076c14afe878: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:52:37.457185 22102 raft_consensus.cc:482] T 40d369f545c243c1a3e1d39e88f7aa78 P d8f2b374859b449da01967e60ecea5aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:52:37.457329 22102 raft_consensus.cc:504] T 40d369f545c243c1a3e1d39e88f7aa78 P d8f2b374859b449da01967e60ecea5aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.457790 22102 leader_election.cc:288] T 40d369f545c243c1a3e1d39e88f7aa78 P d8f2b374859b449da01967e60ecea5aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0989b9cc02824c4fb1902322ef469ed2 (127.8.154.2:42803), 1b449493e13e45b78f81076c14afe878 (127.8.154.1:40933)
I20260809 08:52:37.460716 22086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:37.458081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:54405 (local address 127.8.154.2:42803)
0809 08:52:37.458130 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:37.458132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:37.458143 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:52:37.458247 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:37.458248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:37.458264 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:37.458334 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:37.458493 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.458971 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.459926 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.460358 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.460441 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:37.460452 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:37.460606 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 08:52:37.460642 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 08:52:37.460821 22088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:37.458054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:48171 (local address 127.8.154.1:40933)
0809 08:52:37.458100 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:37.458102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:37.458373 (+ 271us) server_negotiation.cc:407] Connection header received
0809 08:52:37.458450 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:37.458451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:37.458463 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:37.458534 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:37.458701 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.459179 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.460093 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.460454 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.460539 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:37.460549 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:37.460716 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 08:52:37.460752 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 08:52:37.460843 22105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:37.458002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:40933 (local address 127.8.154.3:48171)
0809 08:52:37.458280 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:37.458298 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:37.458382 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:37.458540 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:37.458541 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:37.458542 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:37.458612 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:37.458614 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.459269 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.459270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:37.459978 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:37.459979 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.460555 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.460556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:37.460635 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:37.460638 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:52:37.460704 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 08:52:37.460745 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"mutex_wait_us":70,"thread_start_us":151,"threads_started":1}
I20260809 08:52:37.460737 21877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:37.457996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:42803 (local address 127.8.154.3:54405)
0809 08:52:37.458084 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:37.458098 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:37.458154 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:37.458340 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:37.458341 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:37.458343 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:37.458420 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:37.458423 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.459057 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.459058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:37.459823 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:37.459826 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.460439 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.460440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:37.460528 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:37.460530 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 08:52:37.460601 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 08:52:37.460642 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 08:52:37.461266 21517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40d369f545c243c1a3e1d39e88f7aa78" candidate_uuid: "d8f2b374859b449da01967e60ecea5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b449493e13e45b78f81076c14afe878" is_pre_election: true
I20260809 08:52:37.461443 21517 raft_consensus.cc:2454] T 40d369f545c243c1a3e1d39e88f7aa78 P 1b449493e13e45b78f81076c14afe878 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8f2b374859b449da01967e60ecea5aa in term 0.
I20260809 08:52:37.461443 21752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40d369f545c243c1a3e1d39e88f7aa78" candidate_uuid: "d8f2b374859b449da01967e60ecea5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0989b9cc02824c4fb1902322ef469ed2" is_pre_election: true
I20260809 08:52:37.461613 21752 raft_consensus.cc:2454] T 40d369f545c243c1a3e1d39e88f7aa78 P 0989b9cc02824c4fb1902322ef469ed2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8f2b374859b449da01967e60ecea5aa in term 0.
I20260809 08:52:37.461722 21867 leader_election.cc:302] T 40d369f545c243c1a3e1d39e88f7aa78 P d8f2b374859b449da01967e60ecea5aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b449493e13e45b78f81076c14afe878, d8f2b374859b449da01967e60ecea5aa; no voters:
I20260809 08:52:37.461853 22102 raft_consensus.cc:2781] T 40d369f545c243c1a3e1d39e88f7aa78 P d8f2b374859b449da01967e60ecea5aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:52:37.461910 22102 raft_consensus.cc:482] T 40d369f545c243c1a3e1d39e88f7aa78 P d8f2b374859b449da01967e60ecea5aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:52:37.461933 22102 raft_consensus.cc:3037] T 40d369f545c243c1a3e1d39e88f7aa78 P d8f2b374859b449da01967e60ecea5aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:37.462886 22102 raft_consensus.cc:504] T 40d369f545c243c1a3e1d39e88f7aa78 P d8f2b374859b449da01967e60ecea5aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.463043 22102 leader_election.cc:288] T 40d369f545c243c1a3e1d39e88f7aa78 P d8f2b374859b449da01967e60ecea5aa [CANDIDATE]: Term 1 election: Requested vote from peers 0989b9cc02824c4fb1902322ef469ed2 (127.8.154.2:42803), 1b449493e13e45b78f81076c14afe878 (127.8.154.1:40933)
I20260809 08:52:37.463191 21752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40d369f545c243c1a3e1d39e88f7aa78" candidate_uuid: "d8f2b374859b449da01967e60ecea5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0989b9cc02824c4fb1902322ef469ed2"
I20260809 08:52:37.463203 21517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40d369f545c243c1a3e1d39e88f7aa78" candidate_uuid: "d8f2b374859b449da01967e60ecea5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b449493e13e45b78f81076c14afe878"
I20260809 08:52:37.463290 21752 raft_consensus.cc:3037] T 40d369f545c243c1a3e1d39e88f7aa78 P 0989b9cc02824c4fb1902322ef469ed2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:37.463286 21517 raft_consensus.cc:3037] T 40d369f545c243c1a3e1d39e88f7aa78 P 1b449493e13e45b78f81076c14afe878 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:37.463982 21517 raft_consensus.cc:2454] T 40d369f545c243c1a3e1d39e88f7aa78 P 1b449493e13e45b78f81076c14afe878 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8f2b374859b449da01967e60ecea5aa in term 1.
I20260809 08:52:37.464154 21867 leader_election.cc:302] T 40d369f545c243c1a3e1d39e88f7aa78 P d8f2b374859b449da01967e60ecea5aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b449493e13e45b78f81076c14afe878, d8f2b374859b449da01967e60ecea5aa; no voters:
I20260809 08:52:37.464155 21752 raft_consensus.cc:2454] T 40d369f545c243c1a3e1d39e88f7aa78 P 0989b9cc02824c4fb1902322ef469ed2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8f2b374859b449da01967e60ecea5aa in term 1.
I20260809 08:52:37.464267 22102 raft_consensus.cc:2781] T 40d369f545c243c1a3e1d39e88f7aa78 P d8f2b374859b449da01967e60ecea5aa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:52:37.464452 22102 raft_consensus.cc:686] T 40d369f545c243c1a3e1d39e88f7aa78 P d8f2b374859b449da01967e60ecea5aa [term 1 LEADER]: Becoming Leader. State: Replica: d8f2b374859b449da01967e60ecea5aa, State: Running, Role: LEADER
I20260809 08:52:37.464577 22102 consensus_queue.cc:237] T 40d369f545c243c1a3e1d39e88f7aa78 P d8f2b374859b449da01967e60ecea5aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.465502 21279 catalog_manager.cc:5495] T 40d369f545c243c1a3e1d39e88f7aa78 P d8f2b374859b449da01967e60ecea5aa reported cstate change: term changed from 0 to 1, leader changed from <none> to d8f2b374859b449da01967e60ecea5aa (127.8.154.3). New cstate: current_term: 1 leader_uuid: "d8f2b374859b449da01967e60ecea5aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } health_report { overall_health: HEALTHY } } }
I20260809 08:52:37.472846 22090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:37.470516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42616 (local address 127.8.154.3:41705)
0809 08:52:37.470555 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:37.470556 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:37.470569 (+ 13us) server_negotiation.cc:407] Connection header received
0809 08:52:37.470648 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:37.470649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:37.470665 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:37.470722 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:52:37.470838 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.471280 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.472049 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.472301 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.472357 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:37.472515 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:52:37.472623 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:52:37.472684 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:37.472732 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 08:52:37.472750 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 08:52:37.474594 21752 raft_consensus.cc:1261] T 40d369f545c243c1a3e1d39e88f7aa78 P 0989b9cc02824c4fb1902322ef469ed2 [term 1 FOLLOWER]: Refusing update from remote peer d8f2b374859b449da01967e60ecea5aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:52:37.474628 21517 raft_consensus.cc:1261] T 40d369f545c243c1a3e1d39e88f7aa78 P 1b449493e13e45b78f81076c14afe878 [term 1 FOLLOWER]: Refusing update from remote peer d8f2b374859b449da01967e60ecea5aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:52:37.474939 22102 consensus_queue.cc:1035] T 40d369f545c243c1a3e1d39e88f7aa78 P d8f2b374859b449da01967e60ecea5aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:52:37.475076 22110 consensus_queue.cc:1035] T 40d369f545c243c1a3e1d39e88f7aa78 P d8f2b374859b449da01967e60ecea5aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:52:37.477828 22116 mvcc.cc:205] Tried to move back new op lower bound from 7316543723413561344 to 7316543723375742976. Current Snapshot: MvccSnapshot[applied={T|T < 7316543723413561344 or (T in {7316543723413561344})}]
I20260809 08:52:37.478385 22117 mvcc.cc:205] Tried to move back new op lower bound from 7316543723413561344 to 7316543723375742976. Current Snapshot: MvccSnapshot[applied={T|T < 7316543723413561344 or (T in {7316543723413561344})}]
I20260809 08:52:37.479781 21282 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:52:37.479883 21282 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:52:37.480732 21282 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eea4f50c735c4e06b19ef831930e79b9: Sending DeleteTablet for 3 replicas of tablet 40d369f545c243c1a3e1d39e88f7aa78
I20260809 08:52:37.481050 21496 tablet_service.cc:1514] Processing DeleteTablet for tablet 40d369f545c243c1a3e1d39e88f7aa78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:37 UTC) from {username='oryx', principal='oryx/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:39108
I20260809 08:52:37.481282 21722 tablet_service.cc:1514] Processing DeleteTablet for tablet 40d369f545c243c1a3e1d39e88f7aa78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:37 UTC) from {username='oryx', principal='oryx/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:51234
I20260809 08:52:37.481501 22131 tablet_replica.cc:330] T 40d369f545c243c1a3e1d39e88f7aa78 P 0989b9cc02824c4fb1902322ef469ed2: stopping tablet replica
I20260809 08:52:37.481604 22131 raft_consensus.cc:2229] T 40d369f545c243c1a3e1d39e88f7aa78 P 0989b9cc02824c4fb1902322ef469ed2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:52:37.481585 21940 tablet_service.cc:1514] Processing DeleteTablet for tablet 40d369f545c243c1a3e1d39e88f7aa78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:37 UTC) from {username='oryx', principal='oryx/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:42604
I20260809 08:52:37.481542 21282 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38294:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:52:37.481685 21282 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:52:37.481729 22131 raft_consensus.cc:2258] T 40d369f545c243c1a3e1d39e88f7aa78 P 0989b9cc02824c4fb1902322ef469ed2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:52:37.482089 22128 tablet_replica.cc:330] T 40d369f545c243c1a3e1d39e88f7aa78 P 1b449493e13e45b78f81076c14afe878: stopping tablet replica
I20260809 08:52:37.482193 22128 raft_consensus.cc:2229] T 40d369f545c243c1a3e1d39e88f7aa78 P 1b449493e13e45b78f81076c14afe878 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:52:37.482206 22131 ts_tablet_manager.cc:1905] T 40d369f545c243c1a3e1d39e88f7aa78 P 0989b9cc02824c4fb1902322ef469ed2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:52:37.482297 22128 raft_consensus.cc:2258] T 40d369f545c243c1a3e1d39e88f7aa78 P 1b449493e13e45b78f81076c14afe878 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:52:37.482672 22128 ts_tablet_manager.cc:1905] T 40d369f545c243c1a3e1d39e88f7aa78 P 1b449493e13e45b78f81076c14afe878: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:52:37.483740 22133 tablet_replica.cc:330] T 40d369f545c243c1a3e1d39e88f7aa78 P d8f2b374859b449da01967e60ecea5aa: stopping tablet replica
I20260809 08:52:37.483881 22131 ts_tablet_manager.cc:1918] T 40d369f545c243c1a3e1d39e88f7aa78 P 0989b9cc02824c4fb1902322ef469ed2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:52:37.483917 22133 raft_consensus.cc:2229] T 40d369f545c243c1a3e1d39e88f7aa78 P d8f2b374859b449da01967e60ecea5aa [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:52:37.483959 22131 log.cc:1196] T 40d369f545c243c1a3e1d39e88f7aa78 P 0989b9cc02824c4fb1902322ef469ed2: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-1/wal/wals/40d369f545c243c1a3e1d39e88f7aa78
I20260809 08:52:37.484025 22133 raft_consensus.cc:2258] T 40d369f545c243c1a3e1d39e88f7aa78 P d8f2b374859b449da01967e60ecea5aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:52:37.484302 22131 ts_tablet_manager.cc:1939] T 40d369f545c243c1a3e1d39e88f7aa78 P 0989b9cc02824c4fb1902322ef469ed2: Deleting consensus metadata
I20260809 08:52:37.484496 22133 ts_tablet_manager.cc:1905] T 40d369f545c243c1a3e1d39e88f7aa78 P d8f2b374859b449da01967e60ecea5aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:52:37.484758 21263 catalog_manager.cc:4841] TS 0989b9cc02824c4fb1902322ef469ed2 (127.8.154.2:42803): tablet 40d369f545c243c1a3e1d39e88f7aa78 (table test-table [id=81a66db903564b1da15ec7e75a0003a1]) successfully deleted
I20260809 08:52:37.485180 21940 tablet_service.cc:1467] Processing CreateTablet for tablet 62c8f53eb4e0467ea744279cfcbb7a42 (DEFAULT_TABLE table=test-table [id=bae2a3e7e5fd4abb938bc5194c74341c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:37.485181 21496 tablet_service.cc:1467] Processing CreateTablet for tablet 62c8f53eb4e0467ea744279cfcbb7a42 (DEFAULT_TABLE table=test-table [id=bae2a3e7e5fd4abb938bc5194c74341c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:37.485311 21940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62c8f53eb4e0467ea744279cfcbb7a42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:37.485311 21496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62c8f53eb4e0467ea744279cfcbb7a42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:37.486181 22133 ts_tablet_manager.cc:1918] T 40d369f545c243c1a3e1d39e88f7aa78 P d8f2b374859b449da01967e60ecea5aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:52:37.486261 22133 log.cc:1196] T 40d369f545c243c1a3e1d39e88f7aa78 P d8f2b374859b449da01967e60ecea5aa: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-2/wal/wals/40d369f545c243c1a3e1d39e88f7aa78
I20260809 08:52:37.486536 22133 ts_tablet_manager.cc:1939] T 40d369f545c243c1a3e1d39e88f7aa78 P d8f2b374859b449da01967e60ecea5aa: Deleting consensus metadata
I20260809 08:52:37.486675 22091 tablet_bootstrap.cc:492] T 62c8f53eb4e0467ea744279cfcbb7a42 P d8f2b374859b449da01967e60ecea5aa: Bootstrap starting.
I20260809 08:52:37.486702 22093 tablet_bootstrap.cc:492] T 62c8f53eb4e0467ea744279cfcbb7a42 P 1b449493e13e45b78f81076c14afe878: Bootstrap starting.
I20260809 08:52:37.487097 22093 tablet_bootstrap.cc:654] T 62c8f53eb4e0467ea744279cfcbb7a42 P 1b449493e13e45b78f81076c14afe878: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:37.487097 22091 tablet_bootstrap.cc:654] T 62c8f53eb4e0467ea744279cfcbb7a42 P d8f2b374859b449da01967e60ecea5aa: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:37.487546 21262 catalog_manager.cc:4841] TS d8f2b374859b449da01967e60ecea5aa (127.8.154.3:41705): tablet 40d369f545c243c1a3e1d39e88f7aa78 (table test-table [id=81a66db903564b1da15ec7e75a0003a1]) successfully deleted
I20260809 08:52:37.487829 21722 tablet_service.cc:1467] Processing CreateTablet for tablet 62c8f53eb4e0467ea744279cfcbb7a42 (DEFAULT_TABLE table=test-table [id=bae2a3e7e5fd4abb938bc5194c74341c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:37.487968 21722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62c8f53eb4e0467ea744279cfcbb7a42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:37.488001 22093 tablet_bootstrap.cc:492] T 62c8f53eb4e0467ea744279cfcbb7a42 P 1b449493e13e45b78f81076c14afe878: No bootstrap required, opened a new log
I20260809 08:52:37.488090 22093 ts_tablet_manager.cc:1397] T 62c8f53eb4e0467ea744279cfcbb7a42 P 1b449493e13e45b78f81076c14afe878: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:37.488256 22093 raft_consensus.cc:348] T 62c8f53eb4e0467ea744279cfcbb7a42 P 1b449493e13e45b78f81076c14afe878 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.488335 22093 raft_consensus.cc:374] T 62c8f53eb4e0467ea744279cfcbb7a42 P 1b449493e13e45b78f81076c14afe878 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:37.488364 22093 raft_consensus.cc:729] T 62c8f53eb4e0467ea744279cfcbb7a42 P 1b449493e13e45b78f81076c14afe878 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b449493e13e45b78f81076c14afe878, State: Initialized, Role: FOLLOWER
I20260809 08:52:37.488408 22093 consensus_queue.cc:260] T 62c8f53eb4e0467ea744279cfcbb7a42 P 1b449493e13e45b78f81076c14afe878 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.488523 22093 ts_tablet_manager.cc:1428] T 62c8f53eb4e0467ea744279cfcbb7a42 P 1b449493e13e45b78f81076c14afe878: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:37.489264 22096 tablet_bootstrap.cc:492] T 62c8f53eb4e0467ea744279cfcbb7a42 P 0989b9cc02824c4fb1902322ef469ed2: Bootstrap starting.
I20260809 08:52:37.489635 22091 tablet_bootstrap.cc:492] T 62c8f53eb4e0467ea744279cfcbb7a42 P d8f2b374859b449da01967e60ecea5aa: No bootstrap required, opened a new log
I20260809 08:52:37.489691 22091 ts_tablet_manager.cc:1397] T 62c8f53eb4e0467ea744279cfcbb7a42 P d8f2b374859b449da01967e60ecea5aa: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 08:52:37.489729 22096 tablet_bootstrap.cc:654] T 62c8f53eb4e0467ea744279cfcbb7a42 P 0989b9cc02824c4fb1902322ef469ed2: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:37.489830 22091 raft_consensus.cc:348] T 62c8f53eb4e0467ea744279cfcbb7a42 P d8f2b374859b449da01967e60ecea5aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.489920 22091 raft_consensus.cc:374] T 62c8f53eb4e0467ea744279cfcbb7a42 P d8f2b374859b449da01967e60ecea5aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:37.489943 22091 raft_consensus.cc:729] T 62c8f53eb4e0467ea744279cfcbb7a42 P d8f2b374859b449da01967e60ecea5aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8f2b374859b449da01967e60ecea5aa, State: Initialized, Role: FOLLOWER
I20260809 08:52:37.489992 22091 consensus_queue.cc:260] T 62c8f53eb4e0467ea744279cfcbb7a42 P d8f2b374859b449da01967e60ecea5aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.490089 22091 ts_tablet_manager.cc:1428] T 62c8f53eb4e0467ea744279cfcbb7a42 P d8f2b374859b449da01967e60ecea5aa: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 08:52:37.490427 22096 tablet_bootstrap.cc:492] T 62c8f53eb4e0467ea744279cfcbb7a42 P 0989b9cc02824c4fb1902322ef469ed2: No bootstrap required, opened a new log
I20260809 08:52:37.490477 22096 ts_tablet_manager.cc:1397] T 62c8f53eb4e0467ea744279cfcbb7a42 P 0989b9cc02824c4fb1902322ef469ed2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:37.490635 22096 raft_consensus.cc:348] T 62c8f53eb4e0467ea744279cfcbb7a42 P 0989b9cc02824c4fb1902322ef469ed2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.490662 22128 ts_tablet_manager.cc:1918] T 40d369f545c243c1a3e1d39e88f7aa78 P 1b449493e13e45b78f81076c14afe878: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:52:37.490700 22096 raft_consensus.cc:374] T 62c8f53eb4e0467ea744279cfcbb7a42 P 0989b9cc02824c4fb1902322ef469ed2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:37.490718 22096 raft_consensus.cc:729] T 62c8f53eb4e0467ea744279cfcbb7a42 P 0989b9cc02824c4fb1902322ef469ed2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0989b9cc02824c4fb1902322ef469ed2, State: Initialized, Role: FOLLOWER
I20260809 08:52:37.490753 22096 consensus_queue.cc:260] T 62c8f53eb4e0467ea744279cfcbb7a42 P 0989b9cc02824c4fb1902322ef469ed2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.490810 22096 ts_tablet_manager.cc:1428] T 62c8f53eb4e0467ea744279cfcbb7a42 P 0989b9cc02824c4fb1902322ef469ed2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:37.490736 22128 log.cc:1196] T 40d369f545c243c1a3e1d39e88f7aa78 P 1b449493e13e45b78f81076c14afe878: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-0/wal/wals/40d369f545c243c1a3e1d39e88f7aa78
I20260809 08:52:37.491650 22128 ts_tablet_manager.cc:1939] T 40d369f545c243c1a3e1d39e88f7aa78 P 1b449493e13e45b78f81076c14afe878: Deleting consensus metadata
I20260809 08:52:37.492115 21256 catalog_manager.cc:4841] TS 1b449493e13e45b78f81076c14afe878 (127.8.154.1:40933): tablet 40d369f545c243c1a3e1d39e88f7aa78 (table test-table [id=81a66db903564b1da15ec7e75a0003a1]) successfully deleted
I20260809 08:52:37.515363 22103 raft_consensus.cc:482] T 62c8f53eb4e0467ea744279cfcbb7a42 P 1b449493e13e45b78f81076c14afe878 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:52:37.515528 22103 raft_consensus.cc:504] T 62c8f53eb4e0467ea744279cfcbb7a42 P 1b449493e13e45b78f81076c14afe878 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.515941 22103 leader_election.cc:288] T 62c8f53eb4e0467ea744279cfcbb7a42 P 1b449493e13e45b78f81076c14afe878 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0989b9cc02824c4fb1902322ef469ed2 (127.8.154.2:42803), d8f2b374859b449da01967e60ecea5aa (127.8.154.3:41705)
I20260809 08:52:37.519513 22086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:37.516178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:36811 (local address 127.8.154.2:42803)
0809 08:52:37.516237 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:37.516240 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:37.516349 (+ 109us) server_negotiation.cc:407] Connection header received
0809 08:52:37.516461 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:37.516463 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:37.516476 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:37.516537 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:37.516847 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.517590 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.518707 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.519124 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.519206 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:37.519218 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:37.519406 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 08:52:37.519446 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 08:52:37.519550 22139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:37.516128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:42803 (local address 127.8.154.1:36811)
0809 08:52:37.516281 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:37.516292 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:37.516354 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:37.516665 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:37.516666 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:37.516668 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:37.516747 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:37.516748 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.517711 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.517712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:37.518606 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:37.518608 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.519233 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.519234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:37.519321 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:37.519325 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:52:37.519384 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 08:52:37.519426 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":67,"thread_start_us":80,"threads_started":1}
I20260809 08:52:37.519771 21752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62c8f53eb4e0467ea744279cfcbb7a42" candidate_uuid: "1b449493e13e45b78f81076c14afe878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0989b9cc02824c4fb1902322ef469ed2" is_pre_election: true
I20260809 08:52:37.519873 21752 raft_consensus.cc:2454] T 62c8f53eb4e0467ea744279cfcbb7a42 P 0989b9cc02824c4fb1902322ef469ed2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b449493e13e45b78f81076c14afe878 in term 0.
I20260809 08:52:37.520051 22090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:37.516189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:39075 (local address 127.8.154.3:41705)
0809 08:52:37.516237 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:37.516240 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:37.516475 (+ 235us) server_negotiation.cc:407] Connection header received
0809 08:52:37.516557 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:37.516558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:37.516568 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:37.516619 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:52:37.516817 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.517590 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.519089 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.519631 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.519704 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:37.519714 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:37.519939 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 08:52:37.519978 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 08:52:37.520088 21424 leader_election.cc:302] T 62c8f53eb4e0467ea744279cfcbb7a42 P 1b449493e13e45b78f81076c14afe878 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0989b9cc02824c4fb1902322ef469ed2, 1b449493e13e45b78f81076c14afe878; no voters:
I20260809 08:52:37.520229 22103 raft_consensus.cc:2781] T 62c8f53eb4e0467ea744279cfcbb7a42 P 1b449493e13e45b78f81076c14afe878 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:52:37.520292 22103 raft_consensus.cc:482] T 62c8f53eb4e0467ea744279cfcbb7a42 P 1b449493e13e45b78f81076c14afe878 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:52:37.520324 22103 raft_consensus.cc:3037] T 62c8f53eb4e0467ea744279cfcbb7a42 P 1b449493e13e45b78f81076c14afe878 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:37.520391 22140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:37.516278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:41705 (local address 127.8.154.1:39075)
0809 08:52:37.516411 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:37.516420 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:37.516485 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:37.516645 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:37.516646 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:37.516647 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:52:37.516714 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:37.516717 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.517684 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.517685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:37.518986 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:37.518989 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.519762 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.519764 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:37.519856 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:37.519860 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:52:37.520192 (+ 332us) client_negotiation.cc:240] Negotiation successful
0809 08:52:37.520239 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":29,"thread_start_us":90,"threads_started":1}
I20260809 08:52:37.520658 21965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62c8f53eb4e0467ea744279cfcbb7a42" candidate_uuid: "1b449493e13e45b78f81076c14afe878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8f2b374859b449da01967e60ecea5aa" is_pre_election: true
I20260809 08:52:37.520781 21965 raft_consensus.cc:2454] T 62c8f53eb4e0467ea744279cfcbb7a42 P d8f2b374859b449da01967e60ecea5aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b449493e13e45b78f81076c14afe878 in term 0.
I20260809 08:52:37.521152 22103 raft_consensus.cc:504] T 62c8f53eb4e0467ea744279cfcbb7a42 P 1b449493e13e45b78f81076c14afe878 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.521318 22103 leader_election.cc:288] T 62c8f53eb4e0467ea744279cfcbb7a42 P 1b449493e13e45b78f81076c14afe878 [CANDIDATE]: Term 1 election: Requested vote from peers 0989b9cc02824c4fb1902322ef469ed2 (127.8.154.2:42803), d8f2b374859b449da01967e60ecea5aa (127.8.154.3:41705)
I20260809 08:52:37.521440 21752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62c8f53eb4e0467ea744279cfcbb7a42" candidate_uuid: "1b449493e13e45b78f81076c14afe878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0989b9cc02824c4fb1902322ef469ed2"
I20260809 08:52:37.521512 21752 raft_consensus.cc:3037] T 62c8f53eb4e0467ea744279cfcbb7a42 P 0989b9cc02824c4fb1902322ef469ed2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:37.521697 21965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62c8f53eb4e0467ea744279cfcbb7a42" candidate_uuid: "1b449493e13e45b78f81076c14afe878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8f2b374859b449da01967e60ecea5aa"
I20260809 08:52:37.521761 21965 raft_consensus.cc:3037] T 62c8f53eb4e0467ea744279cfcbb7a42 P d8f2b374859b449da01967e60ecea5aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:37.522372 21752 raft_consensus.cc:2454] T 62c8f53eb4e0467ea744279cfcbb7a42 P 0989b9cc02824c4fb1902322ef469ed2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b449493e13e45b78f81076c14afe878 in term 1.
I20260809 08:52:37.522436 21965 raft_consensus.cc:2454] T 62c8f53eb4e0467ea744279cfcbb7a42 P d8f2b374859b449da01967e60ecea5aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b449493e13e45b78f81076c14afe878 in term 1.
I20260809 08:52:37.522528 21424 leader_election.cc:302] T 62c8f53eb4e0467ea744279cfcbb7a42 P 1b449493e13e45b78f81076c14afe878 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0989b9cc02824c4fb1902322ef469ed2, 1b449493e13e45b78f81076c14afe878; no voters:
I20260809 08:52:37.522624 22103 raft_consensus.cc:2781] T 62c8f53eb4e0467ea744279cfcbb7a42 P 1b449493e13e45b78f81076c14afe878 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:52:37.522768 22103 raft_consensus.cc:686] T 62c8f53eb4e0467ea744279cfcbb7a42 P 1b449493e13e45b78f81076c14afe878 [term 1 LEADER]: Becoming Leader. State: Replica: 1b449493e13e45b78f81076c14afe878, State: Running, Role: LEADER
I20260809 08:52:37.522842 22103 consensus_queue.cc:237] T 62c8f53eb4e0467ea744279cfcbb7a42 P 1b449493e13e45b78f81076c14afe878 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.523557 21282 catalog_manager.cc:5495] T 62c8f53eb4e0467ea744279cfcbb7a42 P 1b449493e13e45b78f81076c14afe878 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b449493e13e45b78f81076c14afe878 (127.8.154.1). New cstate: current_term: 1 leader_uuid: "1b449493e13e45b78f81076c14afe878" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } health_report { overall_health: UNKNOWN } } }
W20260809 08:52:37.526729 22043 tablet.cc:2369] T 62c8f53eb4e0467ea744279cfcbb7a42 P d8f2b374859b449da01967e60ecea5aa: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 08:52:37 dist-test-slave-t3q3 krb5kdc[21203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265556, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.62@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM
I20260809 08:52:37.560797 22153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:37.552945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:42005 (local address 127.0.0.1:38308)
0809 08:52:37.553137 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:37.553149 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:37.553246 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:37.553439 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:37.553440 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:37.553736 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:37.553784 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:37.553786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.554562 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.554563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:37.555136 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:37.555138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.555497 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.555498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:37.555550 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:37.556131 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:37.556136 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:37.557765 (+ 1629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:37.559487 (+ 1722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:37.559493 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:37.559495 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:37.559684 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:37.559921 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:37.559921 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:37.559921 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:37.559978 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:37.560304 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:37.560312 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:37.560394 (+ 82us) client_negotiation.cc:769] Sending connection context
0809 08:52:37.560478 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 08:52:37.560606 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":101,"threads_started":1}
I20260809 08:52:37.560877 21369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:37.553140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38308 (local address 127.8.154.62:42005)
0809 08:52:37.553235 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:37.553237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:37.553247 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:52:37.553368 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:37.553369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:37.553380 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:37.553426 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:37.553889 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.554475 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.555227 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.555426 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.555499 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:37.557871 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:37.557873 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:37.557874 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:37.557881 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:37.559401 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:37.559778 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:37.559779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:37.559779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:37.559846 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:37.560048 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:37.560049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:37.560049 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:37.560216 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:37.560293 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:37.560549 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 08:52:37.560699 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":75}
I20260809 08:52:37.563223 21282 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:38308:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 08:52:37.563452 21282 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:52:37.564975 21496 tablet_service.cc:1467] Processing CreateTablet for tablet 10fec0c68e1a42cfb4478f5ea6b8d1eb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9fc5ad8faffd49918b7cde975fb0f986]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 08:52:37.565028 21940 tablet_service.cc:1467] Processing CreateTablet for tablet 10fec0c68e1a42cfb4478f5ea6b8d1eb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9fc5ad8faffd49918b7cde975fb0f986]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 08:52:37.565150 21496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10fec0c68e1a42cfb4478f5ea6b8d1eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:37.565196 21940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10fec0c68e1a42cfb4478f5ea6b8d1eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:37.566299 22093 tablet_bootstrap.cc:492] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 1b449493e13e45b78f81076c14afe878: Bootstrap starting.
I20260809 08:52:37.566406 21722 tablet_service.cc:1467] Processing CreateTablet for tablet 10fec0c68e1a42cfb4478f5ea6b8d1eb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9fc5ad8faffd49918b7cde975fb0f986]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 08:52:37.566552 21722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10fec0c68e1a42cfb4478f5ea6b8d1eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:37.566737 22093 tablet_bootstrap.cc:654] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 1b449493e13e45b78f81076c14afe878: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:37.566756 22091 tablet_bootstrap.cc:492] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P d8f2b374859b449da01967e60ecea5aa: Bootstrap starting.
I20260809 08:52:37.567191 22091 tablet_bootstrap.cc:654] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P d8f2b374859b449da01967e60ecea5aa: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:37.567804 22093 tablet_bootstrap.cc:492] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 1b449493e13e45b78f81076c14afe878: No bootstrap required, opened a new log
I20260809 08:52:37.567852 22093 ts_tablet_manager.cc:1397] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 1b449493e13e45b78f81076c14afe878: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:52:37.568002 22093 raft_consensus.cc:348] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 1b449493e13e45b78f81076c14afe878 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.568105 22093 raft_consensus.cc:374] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 1b449493e13e45b78f81076c14afe878 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:37.568127 22093 raft_consensus.cc:729] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 1b449493e13e45b78f81076c14afe878 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b449493e13e45b78f81076c14afe878, State: Initialized, Role: FOLLOWER
I20260809 08:52:37.568168 22093 consensus_queue.cc:260] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 1b449493e13e45b78f81076c14afe878 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.568261 22093 ts_tablet_manager.cc:1428] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 1b449493e13e45b78f81076c14afe878: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:37.568370 22091 tablet_bootstrap.cc:492] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P d8f2b374859b449da01967e60ecea5aa: No bootstrap required, opened a new log
I20260809 08:52:37.568423 22091 ts_tablet_manager.cc:1397] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P d8f2b374859b449da01967e60ecea5aa: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:52:37.568419 22103 tablet_replica.cc:439] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 1b449493e13e45b78f81076c14afe878: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } } }
I20260809 08:52:37.568498 22103 tablet_replica.cc:442] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 1b449493e13e45b78f81076c14afe878: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:52:37.568588 22096 tablet_bootstrap.cc:492] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 0989b9cc02824c4fb1902322ef469ed2: Bootstrap starting.
I20260809 08:52:37.568580 22091 raft_consensus.cc:348] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P d8f2b374859b449da01967e60ecea5aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.568675 22091 raft_consensus.cc:374] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P d8f2b374859b449da01967e60ecea5aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:37.568706 22091 raft_consensus.cc:729] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P d8f2b374859b449da01967e60ecea5aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8f2b374859b449da01967e60ecea5aa, State: Initialized, Role: FOLLOWER
I20260809 08:52:37.568746 22091 consensus_queue.cc:260] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P d8f2b374859b449da01967e60ecea5aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.568842 22091 ts_tablet_manager.cc:1428] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P d8f2b374859b449da01967e60ecea5aa: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:37.568893 22121 tablet_replica.cc:439] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P d8f2b374859b449da01967e60ecea5aa: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } } }
I20260809 08:52:37.568996 22121 tablet_replica.cc:442] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P d8f2b374859b449da01967e60ecea5aa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:52:37.569024 22096 tablet_bootstrap.cc:654] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 0989b9cc02824c4fb1902322ef469ed2: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:37.569921 22096 tablet_bootstrap.cc:492] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 0989b9cc02824c4fb1902322ef469ed2: No bootstrap required, opened a new log
I20260809 08:52:37.569964 22096 ts_tablet_manager.cc:1397] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 0989b9cc02824c4fb1902322ef469ed2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:37.570118 22096 raft_consensus.cc:348] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 0989b9cc02824c4fb1902322ef469ed2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.570173 22096 raft_consensus.cc:374] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 0989b9cc02824c4fb1902322ef469ed2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:37.570185 22096 raft_consensus.cc:729] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 0989b9cc02824c4fb1902322ef469ed2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0989b9cc02824c4fb1902322ef469ed2, State: Initialized, Role: FOLLOWER
I20260809 08:52:37.570235 22096 consensus_queue.cc:260] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 0989b9cc02824c4fb1902322ef469ed2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.570329 22096 ts_tablet_manager.cc:1428] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 0989b9cc02824c4fb1902322ef469ed2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:37.570382 22104 tablet_replica.cc:439] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 0989b9cc02824c4fb1902322ef469ed2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } } }
I20260809 08:52:37.570465 22104 tablet_replica.cc:442] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 0989b9cc02824c4fb1902322ef469ed2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:52:37.662921 22103 raft_consensus.cc:482] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 1b449493e13e45b78f81076c14afe878 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:52:37.663024 22103 raft_consensus.cc:504] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 1b449493e13e45b78f81076c14afe878 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.663197 22103 leader_election.cc:288] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 1b449493e13e45b78f81076c14afe878 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0989b9cc02824c4fb1902322ef469ed2 (127.8.154.2:42803), d8f2b374859b449da01967e60ecea5aa (127.8.154.3:41705)
I20260809 08:52:37.663503 21965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10fec0c68e1a42cfb4478f5ea6b8d1eb" candidate_uuid: "1b449493e13e45b78f81076c14afe878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8f2b374859b449da01967e60ecea5aa" is_pre_election: true
I20260809 08:52:37.663504 21752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10fec0c68e1a42cfb4478f5ea6b8d1eb" candidate_uuid: "1b449493e13e45b78f81076c14afe878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0989b9cc02824c4fb1902322ef469ed2" is_pre_election: true
I20260809 08:52:37.663628 21752 raft_consensus.cc:2454] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 0989b9cc02824c4fb1902322ef469ed2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b449493e13e45b78f81076c14afe878 in term 0.
I20260809 08:52:37.663628 21965 raft_consensus.cc:2454] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P d8f2b374859b449da01967e60ecea5aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b449493e13e45b78f81076c14afe878 in term 0.
I20260809 08:52:37.663827 21424 leader_election.cc:302] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 1b449493e13e45b78f81076c14afe878 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0989b9cc02824c4fb1902322ef469ed2, 1b449493e13e45b78f81076c14afe878; no voters:
I20260809 08:52:37.663955 22103 raft_consensus.cc:2781] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 1b449493e13e45b78f81076c14afe878 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:52:37.664032 22103 raft_consensus.cc:482] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 1b449493e13e45b78f81076c14afe878 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:52:37.664093 22103 raft_consensus.cc:3037] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 1b449493e13e45b78f81076c14afe878 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:37.664803 22103 raft_consensus.cc:504] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 1b449493e13e45b78f81076c14afe878 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.664952 22103 leader_election.cc:288] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 1b449493e13e45b78f81076c14afe878 [CANDIDATE]: Term 1 election: Requested vote from peers 0989b9cc02824c4fb1902322ef469ed2 (127.8.154.2:42803), d8f2b374859b449da01967e60ecea5aa (127.8.154.3:41705)
I20260809 08:52:37.665099 21965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10fec0c68e1a42cfb4478f5ea6b8d1eb" candidate_uuid: "1b449493e13e45b78f81076c14afe878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8f2b374859b449da01967e60ecea5aa"
I20260809 08:52:37.665181 21965 raft_consensus.cc:3037] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P d8f2b374859b449da01967e60ecea5aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:37.665126 21752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10fec0c68e1a42cfb4478f5ea6b8d1eb" candidate_uuid: "1b449493e13e45b78f81076c14afe878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0989b9cc02824c4fb1902322ef469ed2"
I20260809 08:52:37.665234 21752 raft_consensus.cc:3037] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 0989b9cc02824c4fb1902322ef469ed2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:37.665809 21752 raft_consensus.cc:2454] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 0989b9cc02824c4fb1902322ef469ed2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b449493e13e45b78f81076c14afe878 in term 1.
I20260809 08:52:37.665817 21965 raft_consensus.cc:2454] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P d8f2b374859b449da01967e60ecea5aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b449493e13e45b78f81076c14afe878 in term 1.
I20260809 08:52:37.665984 21424 leader_election.cc:302] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 1b449493e13e45b78f81076c14afe878 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0989b9cc02824c4fb1902322ef469ed2, 1b449493e13e45b78f81076c14afe878; no voters:
I20260809 08:52:37.666092 22103 raft_consensus.cc:2781] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 1b449493e13e45b78f81076c14afe878 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:52:37.666177 22103 raft_consensus.cc:686] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 1b449493e13e45b78f81076c14afe878 [term 1 LEADER]: Becoming Leader. State: Replica: 1b449493e13e45b78f81076c14afe878, State: Running, Role: LEADER
I20260809 08:52:37.666239 22103 consensus_queue.cc:237] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 1b449493e13e45b78f81076c14afe878 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.666585 22142 tablet_replica.cc:439] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 1b449493e13e45b78f81076c14afe878: TxnStatusTablet state changed. Reason: New leader 1b449493e13e45b78f81076c14afe878. Latest consensus state: current_term: 1 leader_uuid: "1b449493e13e45b78f81076c14afe878" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } } }
I20260809 08:52:37.666683 22142 tablet_replica.cc:442] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 1b449493e13e45b78f81076c14afe878: This TxnStatusTablet replica's current role is: LEADER
I20260809 08:52:37.666903 22163 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 08:52:37.667008 22163 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 08:52:37.666999 21279 catalog_manager.cc:5495] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 1b449493e13e45b78f81076c14afe878 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b449493e13e45b78f81076c14afe878 (127.8.154.1). New cstate: current_term: 1 leader_uuid: "1b449493e13e45b78f81076c14afe878" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } health_report { overall_health: UNKNOWN } } }
I20260809 08:52:37.688868 22088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:37.685944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39124 (local address 127.8.154.1:40933)
0809 08:52:37.685986 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:37.685988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:37.686007 (+ 19us) server_negotiation.cc:407] Connection header received
0809 08:52:37.686102 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:37.686102 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:37.686118 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:37.686180 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:52:37.686357 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.686826 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.687940 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.688222 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.688290 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:37.688504 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:52:37.688602 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:52:37.688709 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:37.688770 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 08:52:37.688790 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 08:52:37.688902 22153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:37.685872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:40933 (local address 127.0.0.1:39124)
0809 08:52:37.685961 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:37.685972 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:37.686017 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:37.686216 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:37.686217 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:37.686219 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 08:52:37.686285 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:37.686287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.686909 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.686910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:37.687861 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:37.687863 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.688309 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.688310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:37.688383 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:37.688401 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 08:52:37.688696 (+ 295us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 08:52:37.688705 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 08:52:37.688765 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 08:52:37.688818 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 08:52:37.690047 21752 raft_consensus.cc:1261] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 0989b9cc02824c4fb1902322ef469ed2 [term 1 FOLLOWER]: Refusing update from remote peer 1b449493e13e45b78f81076c14afe878: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:52:37.690152 21965 raft_consensus.cc:1261] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P d8f2b374859b449da01967e60ecea5aa [term 1 FOLLOWER]: Refusing update from remote peer 1b449493e13e45b78f81076c14afe878: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:52:37.690322 22142 consensus_queue.cc:1035] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 1b449493e13e45b78f81076c14afe878 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:52:37.690438 22103 consensus_queue.cc:1035] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 1b449493e13e45b78f81076c14afe878 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:52:37.691062 22104 tablet_replica.cc:439] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 0989b9cc02824c4fb1902322ef469ed2: TxnStatusTablet state changed. Reason: New leader 1b449493e13e45b78f81076c14afe878. Latest consensus state: current_term: 1 leader_uuid: "1b449493e13e45b78f81076c14afe878" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } } }
I20260809 08:52:37.691159 22104 tablet_replica.cc:442] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 0989b9cc02824c4fb1902322ef469ed2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:52:37.691181 22121 tablet_replica.cc:439] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P d8f2b374859b449da01967e60ecea5aa: TxnStatusTablet state changed. Reason: New leader 1b449493e13e45b78f81076c14afe878. Latest consensus state: current_term: 1 leader_uuid: "1b449493e13e45b78f81076c14afe878" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } } }
I20260809 08:52:37.691267 22121 tablet_replica.cc:442] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P d8f2b374859b449da01967e60ecea5aa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:52:37.691833 22103 tablet_replica.cc:439] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 1b449493e13e45b78f81076c14afe878: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1b449493e13e45b78f81076c14afe878" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } } }
I20260809 08:52:37.691905 22103 tablet_replica.cc:442] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 1b449493e13e45b78f81076c14afe878: This TxnStatusTablet replica's current role is: LEADER
I20260809 08:52:37.692631 22115 mvcc.cc:205] Tried to move back new op lower bound from 7316543724296413184 to 7316543724201336832. Current Snapshot: MvccSnapshot[applied={T|T < 7316543724296413184}]
I20260809 08:52:37.692723 22121 tablet_replica.cc:439] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P d8f2b374859b449da01967e60ecea5aa: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1b449493e13e45b78f81076c14afe878" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } } }
I20260809 08:52:37.692782 22121 tablet_replica.cc:442] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P d8f2b374859b449da01967e60ecea5aa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:52:37.693601 22142 tablet_replica.cc:439] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 1b449493e13e45b78f81076c14afe878: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1b449493e13e45b78f81076c14afe878" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } } }
I20260809 08:52:37.693701 22142 tablet_replica.cc:442] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 1b449493e13e45b78f81076c14afe878: This TxnStatusTablet replica's current role is: LEADER
I20260809 08:52:37.694070 22104 tablet_replica.cc:439] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 0989b9cc02824c4fb1902322ef469ed2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1b449493e13e45b78f81076c14afe878" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } } }
I20260809 08:52:37.694157 22104 tablet_replica.cc:442] T 10fec0c68e1a42cfb4478f5ea6b8d1eb P 0989b9cc02824c4fb1902322ef469ed2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:52:37.695981 22088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:37.693327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39136 (local address 127.8.154.1:40933)
0809 08:52:37.693369 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:37.693371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:37.693381 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:52:37.693454 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:37.693455 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:37.693465 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:37.693509 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:52:37.693683 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.694343 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.695388 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.695560 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.695614 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:37.695779 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:52:37.695805 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:52:37.695843 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:37.695898 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 08:52:37.695913 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 08:52:37.701362 22088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:37.698599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:35759 (local address 127.8.154.1:40933)
0809 08:52:37.698640 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:37.698642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:37.698810 (+ 168us) server_negotiation.cc:407] Connection header received
0809 08:52:37.698854 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:37.698855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:37.698862 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:37.698908 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:52:37.699047 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.699488 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.700661 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.700835 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.700915 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:37.701096 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:52:37.701129 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:52:37.701190 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:37.701249 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 08:52:37.701283 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":13,"server-negotiator.queue_time_us":21}
I20260809 08:52:37.701529 22176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:37.698532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:40933 (local address 127.8.154.1:35759)
0809 08:52:37.698737 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:37.698747 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:37.698806 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:37.698915 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:37.698915 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:37.698917 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 08:52:37.698983 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:37.698985 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.699570 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.699570 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:37.700578 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:37.700581 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.700915 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.700915 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:37.700977 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:37.701000 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 08:52:37.701190 (+ 190us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 08:52:37.701192 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 08:52:37.701245 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 08:52:37.701300 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":2,"thread_start_us":90,"threads_started":1}
I20260809 08:52:37.703753 21965 raft_consensus.cc:1261] T 62c8f53eb4e0467ea744279cfcbb7a42 P d8f2b374859b449da01967e60ecea5aa [term 1 FOLLOWER]: Refusing update from remote peer 1b449493e13e45b78f81076c14afe878: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:52:37.703769 21752 raft_consensus.cc:1261] T 62c8f53eb4e0467ea744279cfcbb7a42 P 0989b9cc02824c4fb1902322ef469ed2 [term 1 FOLLOWER]: Refusing update from remote peer 1b449493e13e45b78f81076c14afe878: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:52:37.703966 22169 consensus_queue.cc:1035] T 62c8f53eb4e0467ea744279cfcbb7a42 P 1b449493e13e45b78f81076c14afe878 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:52:37.704108 22142 consensus_queue.cc:1035] T 62c8f53eb4e0467ea744279cfcbb7a42 P 1b449493e13e45b78f81076c14afe878 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:52:37.716144 21282 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:52:37.716253 21282 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:52:37.716917 21282 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eea4f50c735c4e06b19ef831930e79b9: Sending DeleteTablet for 3 replicas of tablet 62c8f53eb4e0467ea744279cfcbb7a42
I20260809 08:52:37.717183 21940 tablet_service.cc:1514] Processing DeleteTablet for tablet 62c8f53eb4e0467ea744279cfcbb7a42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:37 UTC) from {username='oryx', principal='oryx/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:42604
I20260809 08:52:37.717300 22133 tablet_replica.cc:330] T 62c8f53eb4e0467ea744279cfcbb7a42 P d8f2b374859b449da01967e60ecea5aa: stopping tablet replica
I20260809 08:52:37.717298 21722 tablet_service.cc:1514] Processing DeleteTablet for tablet 62c8f53eb4e0467ea744279cfcbb7a42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:37 UTC) from {username='oryx', principal='oryx/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:51234
I20260809 08:52:37.717370 22133 raft_consensus.cc:2229] T 62c8f53eb4e0467ea744279cfcbb7a42 P d8f2b374859b449da01967e60ecea5aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:52:37.717382 22131 tablet_replica.cc:330] T 62c8f53eb4e0467ea744279cfcbb7a42 P 0989b9cc02824c4fb1902322ef469ed2: stopping tablet replica
I20260809 08:52:37.717410 22133 raft_consensus.cc:2258] T 62c8f53eb4e0467ea744279cfcbb7a42 P d8f2b374859b449da01967e60ecea5aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:52:37.717422 22131 raft_consensus.cc:2229] T 62c8f53eb4e0467ea744279cfcbb7a42 P 0989b9cc02824c4fb1902322ef469ed2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:52:37.717458 22131 raft_consensus.cc:2258] T 62c8f53eb4e0467ea744279cfcbb7a42 P 0989b9cc02824c4fb1902322ef469ed2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:52:37.717840 22131 ts_tablet_manager.cc:1905] T 62c8f53eb4e0467ea744279cfcbb7a42 P 0989b9cc02824c4fb1902322ef469ed2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:52:37.719877 22131 ts_tablet_manager.cc:1918] T 62c8f53eb4e0467ea744279cfcbb7a42 P 0989b9cc02824c4fb1902322ef469ed2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:52:37.719935 22131 log.cc:1196] T 62c8f53eb4e0467ea744279cfcbb7a42 P 0989b9cc02824c4fb1902322ef469ed2: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-1/wal/wals/62c8f53eb4e0467ea744279cfcbb7a42
I20260809 08:52:37.720201 21494 tablet_service.cc:1514] Processing DeleteTablet for tablet 62c8f53eb4e0467ea744279cfcbb7a42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:37 UTC) from {username='oryx', principal='oryx/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:39108
I20260809 08:52:37.720337 22128 tablet_replica.cc:330] T 62c8f53eb4e0467ea744279cfcbb7a42 P 1b449493e13e45b78f81076c14afe878: stopping tablet replica
I20260809 08:52:37.720400 22128 raft_consensus.cc:2229] T 62c8f53eb4e0467ea744279cfcbb7a42 P 1b449493e13e45b78f81076c14afe878 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:52:37.720471 22131 ts_tablet_manager.cc:1939] T 62c8f53eb4e0467ea744279cfcbb7a42 P 0989b9cc02824c4fb1902322ef469ed2: Deleting consensus metadata
I20260809 08:52:37.720494 22128 raft_consensus.cc:2258] T 62c8f53eb4e0467ea744279cfcbb7a42 P 1b449493e13e45b78f81076c14afe878 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:52:37.720803 21263 catalog_manager.cc:4841] TS 0989b9cc02824c4fb1902322ef469ed2 (127.8.154.2:42803): tablet 62c8f53eb4e0467ea744279cfcbb7a42 (table test-table [id=bae2a3e7e5fd4abb938bc5194c74341c]) successfully deleted
I20260809 08:52:37.721071 22133 ts_tablet_manager.cc:1905] T 62c8f53eb4e0467ea744279cfcbb7a42 P d8f2b374859b449da01967e60ecea5aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:52:37.721160 22128 ts_tablet_manager.cc:1905] T 62c8f53eb4e0467ea744279cfcbb7a42 P 1b449493e13e45b78f81076c14afe878: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:52:37.722659 22128 ts_tablet_manager.cc:1918] T 62c8f53eb4e0467ea744279cfcbb7a42 P 1b449493e13e45b78f81076c14afe878: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:52:37.722659 22133 ts_tablet_manager.cc:1918] T 62c8f53eb4e0467ea744279cfcbb7a42 P d8f2b374859b449da01967e60ecea5aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:52:37.722714 22133 log.cc:1196] T 62c8f53eb4e0467ea744279cfcbb7a42 P d8f2b374859b449da01967e60ecea5aa: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-2/wal/wals/62c8f53eb4e0467ea744279cfcbb7a42
I20260809 08:52:37.722725 22128 log.cc:1196] T 62c8f53eb4e0467ea744279cfcbb7a42 P 1b449493e13e45b78f81076c14afe878: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-0/wal/wals/62c8f53eb4e0467ea744279cfcbb7a42
I20260809 08:52:37.722941 22133 ts_tablet_manager.cc:1939] T 62c8f53eb4e0467ea744279cfcbb7a42 P d8f2b374859b449da01967e60ecea5aa: Deleting consensus metadata
I20260809 08:52:37.722941 22128 ts_tablet_manager.cc:1939] T 62c8f53eb4e0467ea744279cfcbb7a42 P 1b449493e13e45b78f81076c14afe878: Deleting consensus metadata
I20260809 08:52:37.723513 21262 catalog_manager.cc:4841] TS d8f2b374859b449da01967e60ecea5aa (127.8.154.3:41705): tablet 62c8f53eb4e0467ea744279cfcbb7a42 (table test-table [id=bae2a3e7e5fd4abb938bc5194c74341c]) successfully deleted
I20260809 08:52:37.723875 21256 catalog_manager.cc:4841] TS 1b449493e13e45b78f81076c14afe878 (127.8.154.1:40933): tablet 62c8f53eb4e0467ea744279cfcbb7a42 (table test-table [id=bae2a3e7e5fd4abb938bc5194c74341c]) successfully deleted
Aug 09 08:52:37 dist-test-slave-t3q3 krb5kdc[21203](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 08:52:37 dist-test-slave-t3q3 krb5kdc[21203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:37 dist-test-slave-t3q3 krb5kdc[21203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
W20260809 08:52:37.742774 21369 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:37.736419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38316 (local address 127.8.154.62:42005)
0809 08:52:37.736498 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:37.736500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:37.736678 (+ 178us) server_negotiation.cc:407] Connection header received
0809 08:52:37.736736 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:37.736737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:37.736748 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:37.736799 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:37.737533 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.738018 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.739124 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.739323 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.739378 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:37.742698 (+ 3320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38316: BlockingRecv error: recv got EOF from 127.0.0.1:38316 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 09 08:52:37 dist-test-slave-t3q3 krb5kdc[21203](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM
I20260809 08:52:37.751132 21369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:37.743578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38326 (local address 127.8.154.62:42005)
0809 08:52:37.743625 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:37.743627 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:37.743863 (+ 236us) server_negotiation.cc:407] Connection header received
0809 08:52:37.743894 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:37.743895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:37.743905 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:37.743974 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:37.744583 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.745301 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.745961 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.746239 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.746328 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:37.748437 (+ 2109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:37.748438 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:37.748438 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:37.748445 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:37.749976 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:37.750314 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:37.750315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:37.750316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:37.750393 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:37.750614 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:37.750615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:37.750615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:37.750782 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:37.750879 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:37.750969 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 08:52:37.751045 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 08:52:37.752841 21282 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38326:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:52:37.753024 21282 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:52:37.754494 21722 tablet_service.cc:1467] Processing CreateTablet for tablet fca7888dfc274b3c9eb1df07f001693d (DEFAULT_TABLE table=test-table [id=c9bd32fee0ab4be8bdd6e553f97407a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:37.754513 21940 tablet_service.cc:1467] Processing CreateTablet for tablet fca7888dfc274b3c9eb1df07f001693d (DEFAULT_TABLE table=test-table [id=c9bd32fee0ab4be8bdd6e553f97407a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:37.754645 21940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fca7888dfc274b3c9eb1df07f001693d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:37.754647 21722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fca7888dfc274b3c9eb1df07f001693d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:37.755997 22096 tablet_bootstrap.cc:492] T fca7888dfc274b3c9eb1df07f001693d P 0989b9cc02824c4fb1902322ef469ed2: Bootstrap starting.
I20260809 08:52:37.756009 22091 tablet_bootstrap.cc:492] T fca7888dfc274b3c9eb1df07f001693d P d8f2b374859b449da01967e60ecea5aa: Bootstrap starting.
I20260809 08:52:37.756032 21494 tablet_service.cc:1467] Processing CreateTablet for tablet fca7888dfc274b3c9eb1df07f001693d (DEFAULT_TABLE table=test-table [id=c9bd32fee0ab4be8bdd6e553f97407a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:37.756173 21494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fca7888dfc274b3c9eb1df07f001693d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:37.756433 22096 tablet_bootstrap.cc:654] T fca7888dfc274b3c9eb1df07f001693d P 0989b9cc02824c4fb1902322ef469ed2: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:37.756450 22091 tablet_bootstrap.cc:654] T fca7888dfc274b3c9eb1df07f001693d P d8f2b374859b449da01967e60ecea5aa: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:37.757176 22096 tablet_bootstrap.cc:492] T fca7888dfc274b3c9eb1df07f001693d P 0989b9cc02824c4fb1902322ef469ed2: No bootstrap required, opened a new log
I20260809 08:52:37.757238 22096 ts_tablet_manager.cc:1397] T fca7888dfc274b3c9eb1df07f001693d P 0989b9cc02824c4fb1902322ef469ed2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:37.757385 22096 raft_consensus.cc:348] T fca7888dfc274b3c9eb1df07f001693d P 0989b9cc02824c4fb1902322ef469ed2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.757462 22096 raft_consensus.cc:374] T fca7888dfc274b3c9eb1df07f001693d P 0989b9cc02824c4fb1902322ef469ed2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:37.757485 22096 raft_consensus.cc:729] T fca7888dfc274b3c9eb1df07f001693d P 0989b9cc02824c4fb1902322ef469ed2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0989b9cc02824c4fb1902322ef469ed2, State: Initialized, Role: FOLLOWER
I20260809 08:52:37.757529 22096 consensus_queue.cc:260] T fca7888dfc274b3c9eb1df07f001693d P 0989b9cc02824c4fb1902322ef469ed2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.757623 22096 ts_tablet_manager.cc:1428] T fca7888dfc274b3c9eb1df07f001693d P 0989b9cc02824c4fb1902322ef469ed2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:37.757815 22093 tablet_bootstrap.cc:492] T fca7888dfc274b3c9eb1df07f001693d P 1b449493e13e45b78f81076c14afe878: Bootstrap starting.
I20260809 08:52:37.758225 22093 tablet_bootstrap.cc:654] T fca7888dfc274b3c9eb1df07f001693d P 1b449493e13e45b78f81076c14afe878: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:37.759150 22091 tablet_bootstrap.cc:492] T fca7888dfc274b3c9eb1df07f001693d P d8f2b374859b449da01967e60ecea5aa: No bootstrap required, opened a new log
I20260809 08:52:37.759198 22091 ts_tablet_manager.cc:1397] T fca7888dfc274b3c9eb1df07f001693d P d8f2b374859b449da01967e60ecea5aa: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 08:52:37.759292 22091 raft_consensus.cc:348] T fca7888dfc274b3c9eb1df07f001693d P d8f2b374859b449da01967e60ecea5aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.759337 22091 raft_consensus.cc:374] T fca7888dfc274b3c9eb1df07f001693d P d8f2b374859b449da01967e60ecea5aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:37.759375 22091 raft_consensus.cc:729] T fca7888dfc274b3c9eb1df07f001693d P d8f2b374859b449da01967e60ecea5aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8f2b374859b449da01967e60ecea5aa, State: Initialized, Role: FOLLOWER
I20260809 08:52:37.759406 22091 consensus_queue.cc:260] T fca7888dfc274b3c9eb1df07f001693d P d8f2b374859b449da01967e60ecea5aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.759464 22091 ts_tablet_manager.cc:1428] T fca7888dfc274b3c9eb1df07f001693d P d8f2b374859b449da01967e60ecea5aa: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:37.760669 22093 tablet_bootstrap.cc:492] T fca7888dfc274b3c9eb1df07f001693d P 1b449493e13e45b78f81076c14afe878: No bootstrap required, opened a new log
I20260809 08:52:37.760725 22093 ts_tablet_manager.cc:1397] T fca7888dfc274b3c9eb1df07f001693d P 1b449493e13e45b78f81076c14afe878: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 08:52:37.760891 22093 raft_consensus.cc:348] T fca7888dfc274b3c9eb1df07f001693d P 1b449493e13e45b78f81076c14afe878 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.760965 22093 raft_consensus.cc:374] T fca7888dfc274b3c9eb1df07f001693d P 1b449493e13e45b78f81076c14afe878 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:37.760994 22093 raft_consensus.cc:729] T fca7888dfc274b3c9eb1df07f001693d P 1b449493e13e45b78f81076c14afe878 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b449493e13e45b78f81076c14afe878, State: Initialized, Role: FOLLOWER
I20260809 08:52:37.761035 22093 consensus_queue.cc:260] T fca7888dfc274b3c9eb1df07f001693d P 1b449493e13e45b78f81076c14afe878 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.761147 22093 ts_tablet_manager.cc:1428] T fca7888dfc274b3c9eb1df07f001693d P 1b449493e13e45b78f81076c14afe878: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:37.787601 22121 raft_consensus.cc:482] T fca7888dfc274b3c9eb1df07f001693d P d8f2b374859b449da01967e60ecea5aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:52:37.787731 22121 raft_consensus.cc:504] T fca7888dfc274b3c9eb1df07f001693d P d8f2b374859b449da01967e60ecea5aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.787921 22121 leader_election.cc:288] T fca7888dfc274b3c9eb1df07f001693d P d8f2b374859b449da01967e60ecea5aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b449493e13e45b78f81076c14afe878 (127.8.154.1:40933), 0989b9cc02824c4fb1902322ef469ed2 (127.8.154.2:42803)
I20260809 08:52:37.788130 21752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fca7888dfc274b3c9eb1df07f001693d" candidate_uuid: "d8f2b374859b449da01967e60ecea5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0989b9cc02824c4fb1902322ef469ed2" is_pre_election: true
I20260809 08:52:37.788269 21752 raft_consensus.cc:2454] T fca7888dfc274b3c9eb1df07f001693d P 0989b9cc02824c4fb1902322ef469ed2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8f2b374859b449da01967e60ecea5aa in term 0.
I20260809 08:52:37.788435 21870 leader_election.cc:302] T fca7888dfc274b3c9eb1df07f001693d P d8f2b374859b449da01967e60ecea5aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0989b9cc02824c4fb1902322ef469ed2, d8f2b374859b449da01967e60ecea5aa; no voters:
I20260809 08:52:37.788558 22121 raft_consensus.cc:2781] T fca7888dfc274b3c9eb1df07f001693d P d8f2b374859b449da01967e60ecea5aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:52:37.788640 22121 raft_consensus.cc:482] T fca7888dfc274b3c9eb1df07f001693d P d8f2b374859b449da01967e60ecea5aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:52:37.788663 22121 raft_consensus.cc:3037] T fca7888dfc274b3c9eb1df07f001693d P d8f2b374859b449da01967e60ecea5aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:37.788659 21517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fca7888dfc274b3c9eb1df07f001693d" candidate_uuid: "d8f2b374859b449da01967e60ecea5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b449493e13e45b78f81076c14afe878" is_pre_election: true
I20260809 08:52:37.788756 21517 raft_consensus.cc:2454] T fca7888dfc274b3c9eb1df07f001693d P 1b449493e13e45b78f81076c14afe878 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8f2b374859b449da01967e60ecea5aa in term 0.
I20260809 08:52:37.789639 22121 raft_consensus.cc:504] T fca7888dfc274b3c9eb1df07f001693d P d8f2b374859b449da01967e60ecea5aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.789788 22121 leader_election.cc:288] T fca7888dfc274b3c9eb1df07f001693d P d8f2b374859b449da01967e60ecea5aa [CANDIDATE]: Term 1 election: Requested vote from peers 1b449493e13e45b78f81076c14afe878 (127.8.154.1:40933), 0989b9cc02824c4fb1902322ef469ed2 (127.8.154.2:42803)
I20260809 08:52:37.789961 21517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fca7888dfc274b3c9eb1df07f001693d" candidate_uuid: "d8f2b374859b449da01967e60ecea5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b449493e13e45b78f81076c14afe878"
I20260809 08:52:37.790045 21517 raft_consensus.cc:3037] T fca7888dfc274b3c9eb1df07f001693d P 1b449493e13e45b78f81076c14afe878 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:37.789999 21752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fca7888dfc274b3c9eb1df07f001693d" candidate_uuid: "d8f2b374859b449da01967e60ecea5aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0989b9cc02824c4fb1902322ef469ed2"
I20260809 08:52:37.790091 21752 raft_consensus.cc:3037] T fca7888dfc274b3c9eb1df07f001693d P 0989b9cc02824c4fb1902322ef469ed2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:37.790750 21752 raft_consensus.cc:2454] T fca7888dfc274b3c9eb1df07f001693d P 0989b9cc02824c4fb1902322ef469ed2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8f2b374859b449da01967e60ecea5aa in term 1.
I20260809 08:52:37.790897 21517 raft_consensus.cc:2454] T fca7888dfc274b3c9eb1df07f001693d P 1b449493e13e45b78f81076c14afe878 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8f2b374859b449da01967e60ecea5aa in term 1.
I20260809 08:52:37.790949 21870 leader_election.cc:302] T fca7888dfc274b3c9eb1df07f001693d P d8f2b374859b449da01967e60ecea5aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0989b9cc02824c4fb1902322ef469ed2, d8f2b374859b449da01967e60ecea5aa; no voters:
I20260809 08:52:37.791112 22121 raft_consensus.cc:2781] T fca7888dfc274b3c9eb1df07f001693d P d8f2b374859b449da01967e60ecea5aa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:52:37.791188 22121 raft_consensus.cc:686] T fca7888dfc274b3c9eb1df07f001693d P d8f2b374859b449da01967e60ecea5aa [term 1 LEADER]: Becoming Leader. State: Replica: d8f2b374859b449da01967e60ecea5aa, State: Running, Role: LEADER
I20260809 08:52:37.791234 22121 consensus_queue.cc:237] T fca7888dfc274b3c9eb1df07f001693d P d8f2b374859b449da01967e60ecea5aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.791875 21280 catalog_manager.cc:5495] T fca7888dfc274b3c9eb1df07f001693d P d8f2b374859b449da01967e60ecea5aa reported cstate change: term changed from 0 to 1, leader changed from <none> to d8f2b374859b449da01967e60ecea5aa (127.8.154.3). New cstate: current_term: 1 leader_uuid: "d8f2b374859b449da01967e60ecea5aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } health_report { overall_health: HEALTHY } } }
I20260809 08:52:37.798585 22090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:37.796031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42626 (local address 127.8.154.3:41705)
0809 08:52:37.796091 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:37.796093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:37.796140 (+ 47us) server_negotiation.cc:407] Connection header received
0809 08:52:37.796222 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:37.796223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:37.796234 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:37.796281 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:52:37.796405 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.796858 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.797948 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.798122 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.798172 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:37.798357 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:52:37.798386 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:52:37.798430 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:37.798494 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 08:52:37.798514 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 08:52:37.799273 21517 raft_consensus.cc:1261] T fca7888dfc274b3c9eb1df07f001693d P 1b449493e13e45b78f81076c14afe878 [term 1 FOLLOWER]: Refusing update from remote peer d8f2b374859b449da01967e60ecea5aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:52:37.799268 21752 raft_consensus.cc:1261] T fca7888dfc274b3c9eb1df07f001693d P 0989b9cc02824c4fb1902322ef469ed2 [term 1 FOLLOWER]: Refusing update from remote peer d8f2b374859b449da01967e60ecea5aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:52:37.799460 22102 consensus_queue.cc:1035] T fca7888dfc274b3c9eb1df07f001693d P d8f2b374859b449da01967e60ecea5aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:52:37.799542 22121 consensus_queue.cc:1035] T fca7888dfc274b3c9eb1df07f001693d P d8f2b374859b449da01967e60ecea5aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:52:37.802616 21280 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:52:37.802718 21280 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:52:37.803447 21280 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eea4f50c735c4e06b19ef831930e79b9: Sending DeleteTablet for 3 replicas of tablet fca7888dfc274b3c9eb1df07f001693d
I20260809 08:52:37.803704 21940 tablet_service.cc:1514] Processing DeleteTablet for tablet fca7888dfc274b3c9eb1df07f001693d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:37 UTC) from {username='oryx', principal='oryx/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:42604
I20260809 08:52:37.803738 21494 tablet_service.cc:1514] Processing DeleteTablet for tablet fca7888dfc274b3c9eb1df07f001693d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:37 UTC) from {username='oryx', principal='oryx/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:39108
I20260809 08:52:37.803839 22133 tablet_replica.cc:330] T fca7888dfc274b3c9eb1df07f001693d P d8f2b374859b449da01967e60ecea5aa: stopping tablet replica
I20260809 08:52:37.803851 21722 tablet_service.cc:1514] Processing DeleteTablet for tablet fca7888dfc274b3c9eb1df07f001693d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:37 UTC) from {username='oryx', principal='oryx/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:51234
I20260809 08:52:37.803891 22133 raft_consensus.cc:2229] T fca7888dfc274b3c9eb1df07f001693d P d8f2b374859b449da01967e60ecea5aa [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:52:37.803946 22131 tablet_replica.cc:330] T fca7888dfc274b3c9eb1df07f001693d P 0989b9cc02824c4fb1902322ef469ed2: stopping tablet replica
I20260809 08:52:37.803951 22133 raft_consensus.cc:2258] T fca7888dfc274b3c9eb1df07f001693d P d8f2b374859b449da01967e60ecea5aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:52:37.804001 22131 raft_consensus.cc:2229] T fca7888dfc274b3c9eb1df07f001693d P 0989b9cc02824c4fb1902322ef469ed2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:52:37.804036 22128 tablet_replica.cc:330] T fca7888dfc274b3c9eb1df07f001693d P 1b449493e13e45b78f81076c14afe878: stopping tablet replica
I20260809 08:52:37.804052 22131 raft_consensus.cc:2258] T fca7888dfc274b3c9eb1df07f001693d P 0989b9cc02824c4fb1902322ef469ed2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:52:37.804122 22128 raft_consensus.cc:2229] T fca7888dfc274b3c9eb1df07f001693d P 1b449493e13e45b78f81076c14afe878 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:52:37.804168 22128 raft_consensus.cc:2258] T fca7888dfc274b3c9eb1df07f001693d P 1b449493e13e45b78f81076c14afe878 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:52:37.804507 22133 ts_tablet_manager.cc:1905] T fca7888dfc274b3c9eb1df07f001693d P d8f2b374859b449da01967e60ecea5aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:52:37.804615 22128 ts_tablet_manager.cc:1905] T fca7888dfc274b3c9eb1df07f001693d P 1b449493e13e45b78f81076c14afe878: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:52:37.804615 22131 ts_tablet_manager.cc:1905] T fca7888dfc274b3c9eb1df07f001693d P 0989b9cc02824c4fb1902322ef469ed2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:52:37.806123 22131 ts_tablet_manager.cc:1918] T fca7888dfc274b3c9eb1df07f001693d P 0989b9cc02824c4fb1902322ef469ed2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:52:37.806123 22133 ts_tablet_manager.cc:1918] T fca7888dfc274b3c9eb1df07f001693d P d8f2b374859b449da01967e60ecea5aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:52:37.806185 22131 log.cc:1196] T fca7888dfc274b3c9eb1df07f001693d P 0989b9cc02824c4fb1902322ef469ed2: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-1/wal/wals/fca7888dfc274b3c9eb1df07f001693d
I20260809 08:52:37.806185 22133 log.cc:1196] T fca7888dfc274b3c9eb1df07f001693d P d8f2b374859b449da01967e60ecea5aa: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-2/wal/wals/fca7888dfc274b3c9eb1df07f001693d
I20260809 08:52:37.806160 21280 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38326:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:52:37.806314 21280 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:52:37.806411 22131 ts_tablet_manager.cc:1939] T fca7888dfc274b3c9eb1df07f001693d P 0989b9cc02824c4fb1902322ef469ed2: Deleting consensus metadata
I20260809 08:52:37.806420 22133 ts_tablet_manager.cc:1939] T fca7888dfc274b3c9eb1df07f001693d P d8f2b374859b449da01967e60ecea5aa: Deleting consensus metadata
I20260809 08:52:37.806752 21262 catalog_manager.cc:4841] TS d8f2b374859b449da01967e60ecea5aa (127.8.154.3:41705): tablet fca7888dfc274b3c9eb1df07f001693d (table test-table [id=c9bd32fee0ab4be8bdd6e553f97407a4]) successfully deleted
I20260809 08:52:37.806869 21263 catalog_manager.cc:4841] TS 0989b9cc02824c4fb1902322ef469ed2 (127.8.154.2:42803): tablet fca7888dfc274b3c9eb1df07f001693d (table test-table [id=c9bd32fee0ab4be8bdd6e553f97407a4]) successfully deleted
I20260809 08:52:37.806892 22128 ts_tablet_manager.cc:1918] T fca7888dfc274b3c9eb1df07f001693d P 1b449493e13e45b78f81076c14afe878: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:52:37.806948 22128 log.cc:1196] T fca7888dfc274b3c9eb1df07f001693d P 1b449493e13e45b78f81076c14afe878: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-0/wal/wals/fca7888dfc274b3c9eb1df07f001693d
I20260809 08:52:37.807207 22128 ts_tablet_manager.cc:1939] T fca7888dfc274b3c9eb1df07f001693d P 1b449493e13e45b78f81076c14afe878: Deleting consensus metadata
I20260809 08:52:37.807526 21256 catalog_manager.cc:4841] TS 1b449493e13e45b78f81076c14afe878 (127.8.154.1:40933): tablet fca7888dfc274b3c9eb1df07f001693d (table test-table [id=c9bd32fee0ab4be8bdd6e553f97407a4]) successfully deleted
I20260809 08:52:37.807672 21722 tablet_service.cc:1467] Processing CreateTablet for tablet d146c832517e4eaebafd467d11040c06 (DEFAULT_TABLE table=test-table [id=b065532792d4408f9156e8ebda501970]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:37.807729 21940 tablet_service.cc:1467] Processing CreateTablet for tablet d146c832517e4eaebafd467d11040c06 (DEFAULT_TABLE table=test-table [id=b065532792d4408f9156e8ebda501970]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:37.807729 21494 tablet_service.cc:1467] Processing CreateTablet for tablet d146c832517e4eaebafd467d11040c06 (DEFAULT_TABLE table=test-table [id=b065532792d4408f9156e8ebda501970]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:52:37.807781 21722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d146c832517e4eaebafd467d11040c06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:37.807832 21494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d146c832517e4eaebafd467d11040c06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:37.807832 21940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d146c832517e4eaebafd467d11040c06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:37.809082 22096 tablet_bootstrap.cc:492] T d146c832517e4eaebafd467d11040c06 P 0989b9cc02824c4fb1902322ef469ed2: Bootstrap starting.
I20260809 08:52:37.809082 22093 tablet_bootstrap.cc:492] T d146c832517e4eaebafd467d11040c06 P 1b449493e13e45b78f81076c14afe878: Bootstrap starting.
I20260809 08:52:37.809548 22093 tablet_bootstrap.cc:654] T d146c832517e4eaebafd467d11040c06 P 1b449493e13e45b78f81076c14afe878: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:37.809825 22096 tablet_bootstrap.cc:654] T d146c832517e4eaebafd467d11040c06 P 0989b9cc02824c4fb1902322ef469ed2: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:37.810124 22091 tablet_bootstrap.cc:492] T d146c832517e4eaebafd467d11040c06 P d8f2b374859b449da01967e60ecea5aa: Bootstrap starting.
I20260809 08:52:37.810379 22091 tablet_bootstrap.cc:654] T d146c832517e4eaebafd467d11040c06 P d8f2b374859b449da01967e60ecea5aa: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:37.810571 22096 tablet_bootstrap.cc:492] T d146c832517e4eaebafd467d11040c06 P 0989b9cc02824c4fb1902322ef469ed2: No bootstrap required, opened a new log
I20260809 08:52:37.810612 22096 ts_tablet_manager.cc:1397] T d146c832517e4eaebafd467d11040c06 P 0989b9cc02824c4fb1902322ef469ed2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:52:37.810765 22096 raft_consensus.cc:348] T d146c832517e4eaebafd467d11040c06 P 0989b9cc02824c4fb1902322ef469ed2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.810834 22096 raft_consensus.cc:374] T d146c832517e4eaebafd467d11040c06 P 0989b9cc02824c4fb1902322ef469ed2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:37.810853 22096 raft_consensus.cc:729] T d146c832517e4eaebafd467d11040c06 P 0989b9cc02824c4fb1902322ef469ed2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0989b9cc02824c4fb1902322ef469ed2, State: Initialized, Role: FOLLOWER
I20260809 08:52:37.810890 22096 consensus_queue.cc:260] T d146c832517e4eaebafd467d11040c06 P 0989b9cc02824c4fb1902322ef469ed2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.810933 22091 tablet_bootstrap.cc:492] T d146c832517e4eaebafd467d11040c06 P d8f2b374859b449da01967e60ecea5aa: No bootstrap required, opened a new log
I20260809 08:52:37.810982 22096 ts_tablet_manager.cc:1428] T d146c832517e4eaebafd467d11040c06 P 0989b9cc02824c4fb1902322ef469ed2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:37.810986 22091 ts_tablet_manager.cc:1397] T d146c832517e4eaebafd467d11040c06 P d8f2b374859b449da01967e60ecea5aa: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260809 08:52:37.811084 22091 raft_consensus.cc:348] T d146c832517e4eaebafd467d11040c06 P d8f2b374859b449da01967e60ecea5aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.811167 22091 raft_consensus.cc:374] T d146c832517e4eaebafd467d11040c06 P d8f2b374859b449da01967e60ecea5aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:37.811198 22091 raft_consensus.cc:729] T d146c832517e4eaebafd467d11040c06 P d8f2b374859b449da01967e60ecea5aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8f2b374859b449da01967e60ecea5aa, State: Initialized, Role: FOLLOWER
I20260809 08:52:37.811249 22091 consensus_queue.cc:260] T d146c832517e4eaebafd467d11040c06 P d8f2b374859b449da01967e60ecea5aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.811339 22091 ts_tablet_manager.cc:1428] T d146c832517e4eaebafd467d11040c06 P d8f2b374859b449da01967e60ecea5aa: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 08:52:37.812332 22093 tablet_bootstrap.cc:492] T d146c832517e4eaebafd467d11040c06 P 1b449493e13e45b78f81076c14afe878: No bootstrap required, opened a new log
I20260809 08:52:37.812398 22093 ts_tablet_manager.cc:1397] T d146c832517e4eaebafd467d11040c06 P 1b449493e13e45b78f81076c14afe878: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 08:52:37.812590 22093 raft_consensus.cc:348] T d146c832517e4eaebafd467d11040c06 P 1b449493e13e45b78f81076c14afe878 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.812664 22093 raft_consensus.cc:374] T d146c832517e4eaebafd467d11040c06 P 1b449493e13e45b78f81076c14afe878 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:37.812688 22093 raft_consensus.cc:729] T d146c832517e4eaebafd467d11040c06 P 1b449493e13e45b78f81076c14afe878 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b449493e13e45b78f81076c14afe878, State: Initialized, Role: FOLLOWER
I20260809 08:52:37.812734 22093 consensus_queue.cc:260] T d146c832517e4eaebafd467d11040c06 P 1b449493e13e45b78f81076c14afe878 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.812832 22093 ts_tablet_manager.cc:1428] T d146c832517e4eaebafd467d11040c06 P 1b449493e13e45b78f81076c14afe878: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 08:52:37.859386 22103 raft_consensus.cc:482] T d146c832517e4eaebafd467d11040c06 P 1b449493e13e45b78f81076c14afe878 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:52:37.859529 22103 raft_consensus.cc:504] T d146c832517e4eaebafd467d11040c06 P 1b449493e13e45b78f81076c14afe878 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.859751 22103 leader_election.cc:288] T d146c832517e4eaebafd467d11040c06 P 1b449493e13e45b78f81076c14afe878 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0989b9cc02824c4fb1902322ef469ed2 (127.8.154.2:42803), d8f2b374859b449da01967e60ecea5aa (127.8.154.3:41705)
I20260809 08:52:37.859979 21965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d146c832517e4eaebafd467d11040c06" candidate_uuid: "1b449493e13e45b78f81076c14afe878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8f2b374859b449da01967e60ecea5aa" is_pre_election: true
I20260809 08:52:37.860011 21752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d146c832517e4eaebafd467d11040c06" candidate_uuid: "1b449493e13e45b78f81076c14afe878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0989b9cc02824c4fb1902322ef469ed2" is_pre_election: true
I20260809 08:52:37.860119 21965 raft_consensus.cc:2454] T d146c832517e4eaebafd467d11040c06 P d8f2b374859b449da01967e60ecea5aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b449493e13e45b78f81076c14afe878 in term 0.
I20260809 08:52:37.860144 21752 raft_consensus.cc:2454] T d146c832517e4eaebafd467d11040c06 P 0989b9cc02824c4fb1902322ef469ed2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b449493e13e45b78f81076c14afe878 in term 0.
I20260809 08:52:37.860304 21423 leader_election.cc:302] T d146c832517e4eaebafd467d11040c06 P 1b449493e13e45b78f81076c14afe878 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b449493e13e45b78f81076c14afe878, d8f2b374859b449da01967e60ecea5aa; no voters:
I20260809 08:52:37.860400 22103 raft_consensus.cc:2781] T d146c832517e4eaebafd467d11040c06 P 1b449493e13e45b78f81076c14afe878 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:52:37.860443 22103 raft_consensus.cc:482] T d146c832517e4eaebafd467d11040c06 P 1b449493e13e45b78f81076c14afe878 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:52:37.860471 22103 raft_consensus.cc:3037] T d146c832517e4eaebafd467d11040c06 P 1b449493e13e45b78f81076c14afe878 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:37.861156 22103 raft_consensus.cc:504] T d146c832517e4eaebafd467d11040c06 P 1b449493e13e45b78f81076c14afe878 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.861320 22103 leader_election.cc:288] T d146c832517e4eaebafd467d11040c06 P 1b449493e13e45b78f81076c14afe878 [CANDIDATE]: Term 1 election: Requested vote from peers 0989b9cc02824c4fb1902322ef469ed2 (127.8.154.2:42803), d8f2b374859b449da01967e60ecea5aa (127.8.154.3:41705)
I20260809 08:52:37.861450 21752 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d146c832517e4eaebafd467d11040c06" candidate_uuid: "1b449493e13e45b78f81076c14afe878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0989b9cc02824c4fb1902322ef469ed2"
I20260809 08:52:37.861543 21752 raft_consensus.cc:3037] T d146c832517e4eaebafd467d11040c06 P 0989b9cc02824c4fb1902322ef469ed2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:37.861523 21965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d146c832517e4eaebafd467d11040c06" candidate_uuid: "1b449493e13e45b78f81076c14afe878" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8f2b374859b449da01967e60ecea5aa"
I20260809 08:52:37.861609 21965 raft_consensus.cc:3037] T d146c832517e4eaebafd467d11040c06 P d8f2b374859b449da01967e60ecea5aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:37.862318 21965 raft_consensus.cc:2454] T d146c832517e4eaebafd467d11040c06 P d8f2b374859b449da01967e60ecea5aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b449493e13e45b78f81076c14afe878 in term 1.
I20260809 08:52:37.862329 21752 raft_consensus.cc:2454] T d146c832517e4eaebafd467d11040c06 P 0989b9cc02824c4fb1902322ef469ed2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b449493e13e45b78f81076c14afe878 in term 1.
I20260809 08:52:37.862516 21423 leader_election.cc:302] T d146c832517e4eaebafd467d11040c06 P 1b449493e13e45b78f81076c14afe878 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b449493e13e45b78f81076c14afe878, d8f2b374859b449da01967e60ecea5aa; no voters:
I20260809 08:52:37.862644 22103 raft_consensus.cc:2781] T d146c832517e4eaebafd467d11040c06 P 1b449493e13e45b78f81076c14afe878 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:52:37.862730 22103 raft_consensus.cc:686] T d146c832517e4eaebafd467d11040c06 P 1b449493e13e45b78f81076c14afe878 [term 1 LEADER]: Becoming Leader. State: Replica: 1b449493e13e45b78f81076c14afe878, State: Running, Role: LEADER
I20260809 08:52:37.862812 22103 consensus_queue.cc:237] T d146c832517e4eaebafd467d11040c06 P 1b449493e13e45b78f81076c14afe878 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } }
I20260809 08:52:37.863368 21280 catalog_manager.cc:5495] T d146c832517e4eaebafd467d11040c06 P 1b449493e13e45b78f81076c14afe878 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b449493e13e45b78f81076c14afe878 (127.8.154.1). New cstate: current_term: 1 leader_uuid: "1b449493e13e45b78f81076c14afe878" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b449493e13e45b78f81076c14afe878" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 } health_report { overall_health: UNKNOWN } } }
I20260809 08:52:37.874919 22088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:37.872429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39148 (local address 127.8.154.1:40933)
0809 08:52:37.872486 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:37.872489 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:37.872498 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:52:37.872570 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:37.872571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:37.872581 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:37.872633 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:52:37.872770 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.873252 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.874296 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:37.874473 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:37.874524 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:37.874690 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:52:37.874718 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:52:37.874755 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:37.874809 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 08:52:37.874825 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 08:52:37.877015 21965 raft_consensus.cc:1261] T d146c832517e4eaebafd467d11040c06 P d8f2b374859b449da01967e60ecea5aa [term 1 FOLLOWER]: Refusing update from remote peer 1b449493e13e45b78f81076c14afe878: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:52:37.877123 21752 raft_consensus.cc:1261] T d146c832517e4eaebafd467d11040c06 P 0989b9cc02824c4fb1902322ef469ed2 [term 1 FOLLOWER]: Refusing update from remote peer 1b449493e13e45b78f81076c14afe878: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:52:37.877218 22142 consensus_queue.cc:1035] T d146c832517e4eaebafd467d11040c06 P 1b449493e13e45b78f81076c14afe878 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8f2b374859b449da01967e60ecea5aa" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 41705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:52:37.877348 22142 consensus_queue.cc:1035] T d146c832517e4eaebafd467d11040c06 P 1b449493e13e45b78f81076c14afe878 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0989b9cc02824c4fb1902322ef469ed2" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 42803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:52:37.889744 21282 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:52:37.889864 21282 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38326:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:52:37.890503 21282 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eea4f50c735c4e06b19ef831930e79b9: Sending DeleteTablet for 3 replicas of tablet d146c832517e4eaebafd467d11040c06
I20260809 08:52:37.890761 21722 tablet_service.cc:1514] Processing DeleteTablet for tablet d146c832517e4eaebafd467d11040c06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:37 UTC) from {username='oryx', principal='oryx/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:51234
I20260809 08:52:37.890827 21940 tablet_service.cc:1514] Processing DeleteTablet for tablet d146c832517e4eaebafd467d11040c06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:37 UTC) from {username='oryx', principal='oryx/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:42604
I20260809 08:52:37.890930 22131 tablet_replica.cc:330] T d146c832517e4eaebafd467d11040c06 P 0989b9cc02824c4fb1902322ef469ed2: stopping tablet replica
I20260809 08:52:37.891017 22131 raft_consensus.cc:2229] T d146c832517e4eaebafd467d11040c06 P 0989b9cc02824c4fb1902322ef469ed2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:52:37.891062 22131 raft_consensus.cc:2258] T d146c832517e4eaebafd467d11040c06 P 0989b9cc02824c4fb1902322ef469ed2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:52:37.891131 22133 tablet_replica.cc:330] T d146c832517e4eaebafd467d11040c06 P d8f2b374859b449da01967e60ecea5aa: stopping tablet replica
I20260809 08:52:37.891184 22133 raft_consensus.cc:2229] T d146c832517e4eaebafd467d11040c06 P d8f2b374859b449da01967e60ecea5aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:52:37.891215 22133 raft_consensus.cc:2258] T d146c832517e4eaebafd467d11040c06 P d8f2b374859b449da01967e60ecea5aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:52:37.891350 21494 tablet_service.cc:1514] Processing DeleteTablet for tablet d146c832517e4eaebafd467d11040c06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:52:37 UTC) from {username='oryx', principal='oryx/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:39108
I20260809 08:52:37.891446 22128 tablet_replica.cc:330] T d146c832517e4eaebafd467d11040c06 P 1b449493e13e45b78f81076c14afe878: stopping tablet replica
I20260809 08:52:37.891491 22128 raft_consensus.cc:2229] T d146c832517e4eaebafd467d11040c06 P 1b449493e13e45b78f81076c14afe878 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:52:37.891491 22131 ts_tablet_manager.cc:1905] T d146c832517e4eaebafd467d11040c06 P 0989b9cc02824c4fb1902322ef469ed2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:52:37.891556 22128 raft_consensus.cc:2258] T d146c832517e4eaebafd467d11040c06 P 1b449493e13e45b78f81076c14afe878 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:52:37.891748 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 21401
I20260809 08:52:37.900318 22131 ts_tablet_manager.cc:1918] T d146c832517e4eaebafd467d11040c06 P 0989b9cc02824c4fb1902322ef469ed2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:52:37.900404 22131 log.cc:1196] T d146c832517e4eaebafd467d11040c06 P 0989b9cc02824c4fb1902322ef469ed2: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-1/wal/wals/d146c832517e4eaebafd467d11040c06
I20260809 08:52:37.900445 22133 ts_tablet_manager.cc:1905] T d146c832517e4eaebafd467d11040c06 P d8f2b374859b449da01967e60ecea5aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:52:37.900673 22131 ts_tablet_manager.cc:1939] T d146c832517e4eaebafd467d11040c06 P 0989b9cc02824c4fb1902322ef469ed2: Deleting consensus metadata
I20260809 08:52:37.901078 21263 catalog_manager.cc:4841] TS 0989b9cc02824c4fb1902322ef469ed2 (127.8.154.2:42803): tablet d146c832517e4eaebafd467d11040c06 (table test-table [id=b065532792d4408f9156e8ebda501970]) successfully deleted
I20260809 08:52:37.901963 22133 ts_tablet_manager.cc:1918] T d146c832517e4eaebafd467d11040c06 P d8f2b374859b449da01967e60ecea5aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:52:37.902031 22133 log.cc:1196] T d146c832517e4eaebafd467d11040c06 P d8f2b374859b449da01967e60ecea5aa: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265519942966-8808-0/minicluster-data/ts-2/wal/wals/d146c832517e4eaebafd467d11040c06
I20260809 08:52:37.902241 22133 ts_tablet_manager.cc:1939] T d146c832517e4eaebafd467d11040c06 P d8f2b374859b449da01967e60ecea5aa: Deleting consensus metadata
I20260809 08:52:37.902581 21262 catalog_manager.cc:4841] TS d8f2b374859b449da01967e60ecea5aa (127.8.154.3:41705): tablet d146c832517e4eaebafd467d11040c06 (table test-table [id=b065532792d4408f9156e8ebda501970]) successfully deleted
W20260809 08:52:37.907862 21256 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.154.1:40933 (error 108)
I20260809 08:52:37.908150 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 21617
I20260809 08:52:37.910355 22228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:37.908298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:40933 (local address 127.0.0.1:39158)
0809 08:52:37.910247 (+ 1949us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:37.910302 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.1:40933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1928,"thread_start_us":39,"threads_started":1}
W20260809 08:52:37.911408 21256 catalog_manager.cc:4584] TS 1b449493e13e45b78f81076c14afe878 (127.8.154.1:40933): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d146c832517e4eaebafd467d11040c06: Network error: Client connection negotiation failed: client connection to 127.8.154.1:40933: connect: Connection refused (error 111)
I20260809 08:52:37.925678 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 21838
I20260809 08:52:37.939563 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 21225
2026-08-09T08:52:37Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2699 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:52:38 dist-test-slave-t3q3 krb5kdc[22233](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:52:38 dist-test-slave-t3q3 krb5kdc[22233](info): set up 2 sockets
Aug 09 08:52:38 dist-test-slave-t3q3 krb5kdc[22233](info): commencing operation
krb5kdc: starting...
W20260809 08:52:39.213377 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.246s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:52:39 dist-test-slave-t3q3 krb5kdc[22233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:52:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:52:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
I20260809 08:52:39.336773 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:33853
--webserver_interface=127.8.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:40797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:33853
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:39.418813 22252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:39.419018 22252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:39.419040 22252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:39.420210 22252 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:52:39.420250 22252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:39.420264 22252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:39.420276 22252 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:52:39.420317 22252 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:52:39.421818 22252 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:40797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:33853
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:33853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:39.422086 22252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:39.422322 22252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:39.427376 22267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:39.427381 22262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:39.427579 22252 server_base.cc:1034] running on GCE node
W20260809 08:52:39.427641 22263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:39.427875 22252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:39.428179 22252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:39.429347 22252 hybrid_clock.cc:648] HybridClock initialized: now 1786265559429328 us; error 31 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T08:52:39Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1486 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:52:39 dist-test-slave-t3q3 krb5kdc[22275](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:52:39 dist-test-slave-t3q3 krb5kdc[22275](info): set up 2 sockets
Aug 09 08:52:39 dist-test-slave-t3q3 krb5kdc[22275](info): commencing operation
krb5kdc: starting...
W20260809 08:52:40.677935 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:52:40 dist-test-slave-t3q3 krb5kdc[22275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265560, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:52:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:52:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
I20260809 08:52:40.807267 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:46079
--webserver_interface=127.8.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:34953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:46079
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:40.890542 22297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:40.890793 22297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:40.890830 22297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:40.891958 22297 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:52:40.892037 22297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:40.892060 22297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:40.892078 22297 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:52:40.892138 22297 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:52:40.893783 22297 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:34953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:46079
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:46079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:40.894076 22297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:40.894320 22297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:40.899354 22307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:40.899413 22297 server_base.cc:1034] running on GCE node
W20260809 08:52:40.899377 22303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:40.899380 22304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:40.899866 22297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:40.900225 22297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:40.901374 22297 hybrid_clock.cc:648] HybridClock initialized: now 1786265560901359 us; error 31 us; skew 500 ppm
Aug 09 08:52:40 dist-test-slave-t3q3 krb5kdc[22275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265560, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:40.907655 22297 init.cc:377] Logged in from keytab as kudu/127.8.154.62@KRBTEST.COM (short username kudu)
I20260809 08:52:40.908195 22297 webserver.cc:460] Webserver started at http://127.8.154.62:36299/ using document root <none> and password file <none>
I20260809 08:52:40.908449 22297 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:40.908520 22297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:40.908679 22297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:40.909839 22297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "3e67e0c196334f3fb9ba76e2a3a177c7"
format_stamp: "Formatted at 2026-08-09 08:52:40 on dist-test-slave-t3q3"
server_key: "dc48273761347a67b6631a5c6dff6c1f"
server_key_iv: "a51b5609b9fdb5903bf1437ed7bbe2d8"
server_key_version: "encryptionkey@0"
I20260809 08:52:40.910231 22297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "3e67e0c196334f3fb9ba76e2a3a177c7"
format_stamp: "Formatted at 2026-08-09 08:52:40 on dist-test-slave-t3q3"
server_key: "dc48273761347a67b6631a5c6dff6c1f"
server_key_iv: "a51b5609b9fdb5903bf1437ed7bbe2d8"
server_key_version: "encryptionkey@0"
I20260809 08:52:40.911626 22297 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:52:40.912465 22318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:40.912638 22297 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:40.912761 22297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "3e67e0c196334f3fb9ba76e2a3a177c7"
format_stamp: "Formatted at 2026-08-09 08:52:40 on dist-test-slave-t3q3"
server_key: "dc48273761347a67b6631a5c6dff6c1f"
server_key_iv: "a51b5609b9fdb5903bf1437ed7bbe2d8"
server_key_version: "encryptionkey@0"
I20260809 08:52:40.912840 22297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:40.940856 22297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:40.943005 22297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:40.943168 22297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:40.947273 22297 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:46079
I20260809 08:52:40.947360 22406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:46079 every 8 connection(s)
I20260809 08:52:40.947726 22297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:52:40.948678 22408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:40.951025 22408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7: Bootstrap starting.
I20260809 08:52:40.951584 22408 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:40.951844 22408 log.cc:824] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:40.952589 22408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7: No bootstrap required, opened a new log
I20260809 08:52:40.952778 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 22297
I20260809 08:52:40.952868 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:52:40.953037 8808 external_mini_cluster.cc:1428] Setting key f6620d1d4b1e504d9c49307647d54635
I20260809 08:52:40.954468 22408 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e67e0c196334f3fb9ba76e2a3a177c7" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46079 } }
I20260809 08:52:40.954640 22408 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:40.954689 22408 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e67e0c196334f3fb9ba76e2a3a177c7, State: Initialized, Role: FOLLOWER
I20260809 08:52:40.954828 22408 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e67e0c196334f3fb9ba76e2a3a177c7" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46079 } }
I20260809 08:52:40.954937 22408 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:52:40.954980 22408 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:52:40.955052 22408 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:40.955966 22408 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e67e0c196334f3fb9ba76e2a3a177c7" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46079 } }
I20260809 08:52:40.956120 22408 leader_election.cc:302] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e67e0c196334f3fb9ba76e2a3a177c7; no voters:
I20260809 08:52:40.956302 22408 leader_election.cc:288] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:52:40.956331 22413 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:52:40.956453 22413 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7 [term 1 LEADER]: Becoming Leader. State: Replica: 3e67e0c196334f3fb9ba76e2a3a177c7, State: Running, Role: LEADER
I20260809 08:52:40.956537 22413 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e67e0c196334f3fb9ba76e2a3a177c7" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46079 } }
I20260809 08:52:40.956568 22408 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:52:40.956902 22413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e67e0c196334f3fb9ba76e2a3a177c7. Latest consensus state: current_term: 1 leader_uuid: "3e67e0c196334f3fb9ba76e2a3a177c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e67e0c196334f3fb9ba76e2a3a177c7" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46079 } } }
I20260809 08:52:40.957003 22413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7 [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:40.956915 22415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e67e0c196334f3fb9ba76e2a3a177c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e67e0c196334f3fb9ba76e2a3a177c7" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46079 } } }
I20260809 08:52:40.957312 22415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7 [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:40.957360 22419 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:52:40.957926 22419 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:52:40.959761 22419 catalog_manager.cc:1347] Generated new cluster ID: 8e3795f4a6494cfe9164586fae9f17f3
I20260809 08:52:40.959816 22419 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 08:52:40 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265560, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:40.965919 22412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:40.953581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55438 (local address 127.8.154.62:46079)
0809 08:52:40.953783 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:40.953794 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:40.953831 (+ 37us) server_negotiation.cc:407] Connection header received
0809 08:52:40.954449 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:40.954457 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:40.954507 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:40.954657 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:40.955431 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:40.956570 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:40.957741 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:40.958504 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:40.958922 (+ 418us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:40.961561 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:40.961565 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:40.961566 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:40.961578 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:40.963611 (+ 2033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:40.964014 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:40.964018 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:40.964018 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:40.964204 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:40.964426 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:40.964426 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:40.964427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:40.964758 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:40.965473 (+ 715us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:40.965555 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 08:52:40.965708 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":85,"threads_started":1}
I20260809 08:52:40.974951 22419 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:52:40.975505 22419 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:52:40.978088 22419 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7: Generated new TSK 0
I20260809 08:52:40.978273 22419 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab.
Entry for principal kudu/127.8.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab.
I20260809 08:52:41.010618 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.1:0
--local_ip_for_outbound_sockets=127.8.154.1
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:46079
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:34953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:41.094851 22438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:41.095069 22438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:41.095094 22438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:41.096300 22438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:41.096345 22438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:41.096372 22438 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.1
I20260809 08:52:41.097836 22438 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:34953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:46079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.1
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:41.098085 22438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:41.098346 22438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:41.103754 22445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:41.103745 22449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:41.103745 22444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:41.104051 22438 server_base.cc:1034] running on GCE node
I20260809 08:52:41.104291 22438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:41.104552 22438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:41.105753 22438 hybrid_clock.cc:648] HybridClock initialized: now 1786265561105740 us; error 39 us; skew 500 ppm
Aug 09 08:52:41 dist-test-slave-t3q3 krb5kdc[22275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265561, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:41.109815 22438 init.cc:377] Logged in from keytab as kudu/127.8.154.1@KRBTEST.COM (short username kudu)
I20260809 08:52:41.110481 22438 webserver.cc:460] Webserver started at http://127.8.154.1:33467/ using document root <none> and password file <none>
I20260809 08:52:41.110780 22438 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:41.110858 22438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:41.111013 22438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:41.112767 22438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-0/data/instance:
uuid: "ab147b29f3be40329e7697140b6182d9"
format_stamp: "Formatted at 2026-08-09 08:52:41 on dist-test-slave-t3q3"
server_key: "ec1854de9b63720f228a847387869232"
server_key_iv: "e07309eb1d0d2a670c76d9b1ee90e5ba"
server_key_version: "encryptionkey@0"
I20260809 08:52:41.113267 22438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance:
uuid: "ab147b29f3be40329e7697140b6182d9"
format_stamp: "Formatted at 2026-08-09 08:52:41 on dist-test-slave-t3q3"
server_key: "ec1854de9b63720f228a847387869232"
server_key_iv: "e07309eb1d0d2a670c76d9b1ee90e5ba"
server_key_version: "encryptionkey@0"
I20260809 08:52:41.116220 22438 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 08:52:41.117321 22457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:41.117578 22438 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:41.117710 22438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-0/wal
uuid: "ab147b29f3be40329e7697140b6182d9"
format_stamp: "Formatted at 2026-08-09 08:52:41 on dist-test-slave-t3q3"
server_key: "ec1854de9b63720f228a847387869232"
server_key_iv: "e07309eb1d0d2a670c76d9b1ee90e5ba"
server_key_version: "encryptionkey@0"
I20260809 08:52:41.117796 22438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:41.125840 22438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:41.127930 22438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:41.128126 22438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:41.128377 22438 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:41.128669 22438 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:41.128701 22438 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:41.128724 22438 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:41.128737 22438 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:41.134567 22438 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.1:40727
I20260809 08:52:41.134654 22625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.1:40727 every 8 connection(s)
I20260809 08:52:41.134985 22438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
I20260809 08:52:41.136615 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 22438
I20260809 08:52:41.136740 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance
I20260809 08:52:41.136930 8808 external_mini_cluster.cc:1428] Setting key c6327ef4b149582508a0ae59adacb818
Aug 09 08:52:41 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265561, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.154.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.2@KRBTEST.COM" created.
I20260809 08:52:41.149442 22412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:41.135799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:43407 (local address 127.8.154.62:46079)
0809 08:52:41.135901 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:41.135907 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:41.136129 (+ 222us) server_negotiation.cc:407] Connection header received
0809 08:52:41.136709 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:41.136710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:41.136727 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:41.136795 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:41.138061 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:41.138566 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:41.139608 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:41.139821 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:41.139902 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:41.145998 (+ 6096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:41.145999 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:41.146000 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:41.146017 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:41.147628 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:41.148156 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:41.148157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:41.148157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:41.148228 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:41.148506 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:41.148507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:41.148507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:41.148622 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:41.148678 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:41.149174 (+ 496us) server_negotiation.cc:299] Negotiation successful
0809 08:52:41.149309 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 08:52:41.149506 22630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:41.135708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.1:43407)
0809 08:52:41.135929 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:41.135973 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:41.136590 (+ 617us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:41.136918 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:41.136922 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:41.137724 (+ 802us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:41.137959 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:41.137964 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:41.138666 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:41.138667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:41.139490 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:41.139492 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:41.139910 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:41.139911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:41.140047 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:41.144398 (+ 4351us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:41.144407 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:41.145884 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:41.147717 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:41.147724 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:41.147726 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:41.147953 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:41.148310 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:41.148311 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:41.148312 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:41.148435 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:41.148843 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:41.148850 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:41.149002 (+ 152us) client_negotiation.cc:769] Sending connection context
0809 08:52:41.149138 (+ 136us) client_negotiation.cc:240] Negotiation successful
0809 08:52:41.149279 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":106,"threads_started":1}
I20260809 08:52:41.150161 22627 heartbeater.cc:347] Connected to a master server at 127.8.154.62:46079
I20260809 08:52:41.150282 22627 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:41.150492 22627 heartbeater.cc:511] Master 127.8.154.62:46079 requested a full tablet report, sending...
I20260809 08:52:41.151096 22347 ts_manager.cc:194] Registered new tserver with Master: ab147b29f3be40329e7697140b6182d9 (127.8.154.1:40727)
I20260809 08:52:41.151741 22347 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.1@KRBTEST.COM'} at 127.8.154.1:43407
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab.
Entry for principal kudu/127.8.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab.
I20260809 08:52:41.165300 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.2:0
--local_ip_for_outbound_sockets=127.8.154.2
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:46079
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:34953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:41.248584 22633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:41.248791 22633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:41.248831 22633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:41.249975 22633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:41.250041 22633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:41.250085 22633 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.2
I20260809 08:52:41.251488 22633 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:34953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:46079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.2
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:41.251736 22633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:41.251955 22633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:41.257022 22643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:41.257038 22645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:41.257261 22633 server_base.cc:1034] running on GCE node
W20260809 08:52:41.257037 22647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:41.257644 22633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:41.257891 22633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:41.259029 22633 hybrid_clock.cc:648] HybridClock initialized: now 1786265561259017 us; error 29 us; skew 500 ppm
Aug 09 08:52:41 dist-test-slave-t3q3 krb5kdc[22275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265561, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:41.261526 22633 init.cc:377] Logged in from keytab as kudu/127.8.154.2@KRBTEST.COM (short username kudu)
I20260809 08:52:41.261934 22633 webserver.cc:460] Webserver started at http://127.8.154.2:35895/ using document root <none> and password file <none>
I20260809 08:52:41.262166 22633 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:41.262210 22633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:41.262352 22633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:41.263365 22633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-1/data/instance:
uuid: "5b4c353ca6824b5bae15f06efa77d53c"
format_stamp: "Formatted at 2026-08-09 08:52:41 on dist-test-slave-t3q3"
server_key: "e260277f9944c52b2aecc1aa7288063e"
server_key_iv: "69a519358a0e60ca36e1ba9890c052e1"
server_key_version: "encryptionkey@0"
I20260809 08:52:41.263731 22633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance:
uuid: "5b4c353ca6824b5bae15f06efa77d53c"
format_stamp: "Formatted at 2026-08-09 08:52:41 on dist-test-slave-t3q3"
server_key: "e260277f9944c52b2aecc1aa7288063e"
server_key_iv: "69a519358a0e60ca36e1ba9890c052e1"
server_key_version: "encryptionkey@0"
I20260809 08:52:41.265116 22633 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:52:41.265836 22657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:41.266008 22633 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:52:41.266137 22633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-1/wal
uuid: "5b4c353ca6824b5bae15f06efa77d53c"
format_stamp: "Formatted at 2026-08-09 08:52:41 on dist-test-slave-t3q3"
server_key: "e260277f9944c52b2aecc1aa7288063e"
server_key_iv: "69a519358a0e60ca36e1ba9890c052e1"
server_key_version: "encryptionkey@0"
I20260809 08:52:41.266214 22633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:41.285993 22633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:41.289227 22633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:41.289438 22633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:41.289705 22633 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:41.290038 22633 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:41.290072 22633 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:41.290094 22633 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:41.290128 22633 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:41.296450 22633 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.2:34043
I20260809 08:52:41.296489 22826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.2:34043 every 8 connection(s)
I20260809 08:52:41.296854 22633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
I20260809 08:52:41.301051 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 22633
I20260809 08:52:41.301215 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance
I20260809 08:52:41.301409 8808 external_mini_cluster.cc:1428] Setting key c84a0d55b36eef0100c6eb8058a22c14
Aug 09 08:52:41 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265561, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:41.306974 22830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:41.297597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.2:58873)
0809 08:52:41.297766 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:41.297810 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:41.298298 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:41.298500 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:41.298504 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:41.299077 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:41.299280 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:41.299285 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:41.299992 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:41.299992 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:41.300903 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:41.300907 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:41.301397 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:41.301398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:41.301536 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:41.302078 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:41.302089 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:41.303730 (+ 1641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:41.305604 (+ 1874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:41.305617 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:41.305619 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:41.305836 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:41.306070 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:41.306071 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:41.306071 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:41.306175 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:41.306480 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:41.306486 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:41.306610 (+ 124us) client_negotiation.cc:769] Sending connection context
0809 08:52:41.306696 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 08:52:41.306816 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":91,"threads_started":1}
I20260809 08:52:41.307010 22412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:41.297641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:58873 (local address 127.8.154.62:46079)
0809 08:52:41.297705 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:41.297708 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:41.297900 (+ 192us) server_negotiation.cc:407] Connection header received
0809 08:52:41.298378 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:41.298379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:41.298388 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:41.298447 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:41.299358 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:41.299899 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:41.300992 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:41.301245 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:41.301327 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:41.303837 (+ 2510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:41.303839 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:41.303840 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:41.303849 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:41.305491 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:41.305917 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:41.305919 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:41.305919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:41.305990 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:41.306245 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:41.306246 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:41.306247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:41.306403 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:41.306477 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:41.306750 (+ 273us) server_negotiation.cc:299] Negotiation successful
0809 08:52:41.306864 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 08:52:41.307618 22828 heartbeater.cc:347] Connected to a master server at 127.8.154.62:46079
I20260809 08:52:41.307732 22828 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:41.307934 22828 heartbeater.cc:511] Master 127.8.154.62:46079 requested a full tablet report, sending...
I20260809 08:52:41.308409 22347 ts_manager.cc:194] Registered new tserver with Master: 5b4c353ca6824b5bae15f06efa77d53c (127.8.154.2:34043)
I20260809 08:52:41.308791 22347 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.2@KRBTEST.COM'} at 127.8.154.2:58873
WARNING: no policy specified for kudu/127.8.154.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab.
Entry for principal kudu/127.8.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab.
I20260809 08:52:41.329083 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.3:0
--local_ip_for_outbound_sockets=127.8.154.3
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:46079
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:34953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:41.413878 22837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:41.414090 22837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:41.414114 22837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:41.415212 22837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:41.415302 22837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:41.415361 22837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.3
I20260809 08:52:41.416930 22837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:34953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:46079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.3
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:41.417124 22837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:41.417299 22837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:41.422514 22843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:41.422564 22842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:41.422724 22845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:41.422724 22837 server_base.cc:1034] running on GCE node
I20260809 08:52:41.422999 22837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:41.423215 22837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:41.424387 22837 hybrid_clock.cc:648] HybridClock initialized: now 1786265561424368 us; error 31 us; skew 500 ppm
Aug 09 08:52:41 dist-test-slave-t3q3 krb5kdc[22275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265561, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:41.426909 22837 init.cc:377] Logged in from keytab as kudu/127.8.154.3@KRBTEST.COM (short username kudu)
I20260809 08:52:41.427373 22837 webserver.cc:460] Webserver started at http://127.8.154.3:33371/ using document root <none> and password file <none>
I20260809 08:52:41.427593 22837 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:41.427635 22837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:41.427726 22837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:41.428895 22837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-2/data/instance:
uuid: "4dcd06121c104956992a40aa6b9754cd"
format_stamp: "Formatted at 2026-08-09 08:52:41 on dist-test-slave-t3q3"
server_key: "50d0c0e76fe27c5a047912f3bda027bf"
server_key_iv: "25eaa8036587060b74df7a524274c06d"
server_key_version: "encryptionkey@0"
I20260809 08:52:41.429257 22837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance:
uuid: "4dcd06121c104956992a40aa6b9754cd"
format_stamp: "Formatted at 2026-08-09 08:52:41 on dist-test-slave-t3q3"
server_key: "50d0c0e76fe27c5a047912f3bda027bf"
server_key_iv: "25eaa8036587060b74df7a524274c06d"
server_key_version: "encryptionkey@0"
I20260809 08:52:41.430543 22837 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 08:52:41.431257 22856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:41.431454 22837 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:52:41.431522 22837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-2/wal
uuid: "4dcd06121c104956992a40aa6b9754cd"
format_stamp: "Formatted at 2026-08-09 08:52:41 on dist-test-slave-t3q3"
server_key: "50d0c0e76fe27c5a047912f3bda027bf"
server_key_iv: "25eaa8036587060b74df7a524274c06d"
server_key_version: "encryptionkey@0"
I20260809 08:52:41.431627 22837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:41.449764 22837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:41.451900 22837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:41.452080 22837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:41.452606 22837 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:41.452982 22837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:41.453035 22837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:41.453060 22837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:41.453107 22837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:41.460235 22837 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.3:42813
I20260809 08:52:41.460282 23023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.3:42813 every 8 connection(s)
I20260809 08:52:41.460638 22837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
I20260809 08:52:41.464390 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 22837
I20260809 08:52:41.464510 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance
I20260809 08:52:41.464705 8808 external_mini_cluster.cc:1428] Setting key 7afaeacd45c856702e5338d9978a0d95
Aug 09 08:52:41 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265561, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:41.470631 22412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:41.461399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:37513 (local address 127.8.154.62:46079)
0809 08:52:41.461466 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:41.461468 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:41.461693 (+ 225us) server_negotiation.cc:407] Connection header received
0809 08:52:41.462209 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:41.462210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:41.462221 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:41.462276 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:41.463211 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:41.463720 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:41.465120 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:41.465451 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:41.465521 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:41.467735 (+ 2214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:41.467736 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:41.467737 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:41.467743 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:41.469200 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:41.469613 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:41.469614 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:41.469614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:41.469677 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:41.469934 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:41.469935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:41.469935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:41.470098 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:41.470177 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:41.470460 (+ 283us) server_negotiation.cc:299] Negotiation successful
0809 08:52:41.470543 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 08:52:41.470831 23027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:41.461369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.3:37513)
0809 08:52:41.461561 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:41.461605 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:41.462098 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:41.462344 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:41.462347 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:41.462925 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:41.463128 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:41.463133 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:41.463804 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:41.463804 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:41.465023 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:41.465026 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:41.465593 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:41.465594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:41.465700 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:41.466214 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:41.466223 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:41.467631 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:41.469300 (+ 1669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:41.469307 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:41.469309 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:41.469532 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:41.469752 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:52:41.469753 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:52:41.469754 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:52:41.469863 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:52:41.470169 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:41.470176 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:41.470316 (+ 140us) client_negotiation.cc:769] Sending connection context
0809 08:52:41.470424 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 08:52:41.470583 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":117,"threads_started":1}
I20260809 08:52:41.471302 23024 heartbeater.cc:347] Connected to a master server at 127.8.154.62:46079
I20260809 08:52:41.471422 23024 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:41.471614 23024 heartbeater.cc:511] Master 127.8.154.62:46079 requested a full tablet report, sending...
I20260809 08:52:41.472007 22347 ts_manager.cc:194] Registered new tserver with Master: 4dcd06121c104956992a40aa6b9754cd (127.8.154.3:42813)
I20260809 08:52:41.472422 22347 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.3@KRBTEST.COM'} at 127.8.154.3:37513
I20260809 08:52:41.476503 8808 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.8.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.1.keytab.
Entry for principal oryx/127.8.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.1.keytab.
WARNING: no policy specified for oryx/127.8.154.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.154.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.2.keytab.
Entry for principal oryx/127.8.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.2.keytab.
WARNING: no policy specified for oryx/127.8.154.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.8.154.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.8.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.3.keytab.
Entry for principal oryx/127.8.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.3.keytab.
I20260809 08:52:41.548816 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 22438
I20260809 08:52:41.565210 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 22633
I20260809 08:52:41.577996 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 22837
I20260809 08:52:41.586133 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 22297
I20260809 08:52:41.599300 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:46079
--webserver_interface=127.8.154.62
--webserver_port=36299
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:34953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:46079
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:41.680047 23040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:41.680354 23040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:41.680392 23040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:41.681527 23040 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:52:41.681602 23040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:41.681627 23040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:41.681650 23040 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:52:41.681679 23040 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:52:41.683161 23040 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:34953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:46079
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:46079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=36299
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23040
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:41.683434 23040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:41.683663 23040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 08:52:41.688539 23040 server_base.cc:1034] running on GCE node
W20260809 08:52:41.688546 23048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:41.688561 23049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:41.688597 23053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:41.689049 23040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:41.689406 23040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:41.690582 23040 hybrid_clock.cc:648] HybridClock initialized: now 1786265561690578 us; error 35 us; skew 500 ppm
Aug 09 08:52:41 dist-test-slave-t3q3 krb5kdc[22275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265561, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:41.693172 23040 init.cc:377] Logged in from keytab as kudu/127.8.154.62@KRBTEST.COM (short username kudu)
I20260809 08:52:41.693545 23040 webserver.cc:460] Webserver started at http://127.8.154.62:36299/ using document root <none> and password file <none>
I20260809 08:52:41.693763 23040 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:41.693815 23040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:41.695106 23040 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:52:41.695749 23066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:41.695946 23040 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:52:41.696033 23040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "3e67e0c196334f3fb9ba76e2a3a177c7"
format_stamp: "Formatted at 2026-08-09 08:52:40 on dist-test-slave-t3q3"
server_key: "dc48273761347a67b6631a5c6dff6c1f"
server_key_iv: "a51b5609b9fdb5903bf1437ed7bbe2d8"
server_key_version: "encryptionkey@0"
I20260809 08:52:41.696405 23040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:41.715247 23040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:41.717521 23040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:41.717631 23040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:41.721479 23040 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:46079
I20260809 08:52:41.721553 23141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:46079 every 8 connection(s)
I20260809 08:52:41.721928 23040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:52:41.723404 23142 sys_catalog.cc:263] Verifying existing consensus state
I20260809 08:52:41.724119 23142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7: Bootstrap starting.
I20260809 08:52:41.724252 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 23040
I20260809 08:52:41.724566 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.1:40727
--local_ip_for_outbound_sockets=127.8.154.1
--tserver_master_addrs=127.8.154.62:46079
--webserver_port=33467
--webserver_interface=127.8.154.1
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:34953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.154.1
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:52:41.733388 23142 log.cc:824] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:41.735024 23142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 08:52:41.735319 23142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7: Bootstrap complete.
I20260809 08:52:41.737640 23142 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e67e0c196334f3fb9ba76e2a3a177c7" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46079 } }
I20260809 08:52:41.737864 23142 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e67e0c196334f3fb9ba76e2a3a177c7, State: Initialized, Role: FOLLOWER
I20260809 08:52:41.738030 23142 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e67e0c196334f3fb9ba76e2a3a177c7" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46079 } }
I20260809 08:52:41.738101 23142 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:52:41.738165 23142 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:52:41.738214 23142 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7 [term 1 FOLLOWER]: Advancing to term 2
I20260809 08:52:41.739387 23142 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e67e0c196334f3fb9ba76e2a3a177c7" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46079 } }
I20260809 08:52:41.739509 23142 leader_election.cc:302] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e67e0c196334f3fb9ba76e2a3a177c7; no voters:
I20260809 08:52:41.739718 23142 leader_election.cc:288] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 08:52:41.739761 23148 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 08:52:41.739892 23148 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7 [term 2 LEADER]: Becoming Leader. State: Replica: 3e67e0c196334f3fb9ba76e2a3a177c7, State: Running, Role: LEADER
I20260809 08:52:41.740000 23142 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:52:41.739979 23148 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e67e0c196334f3fb9ba76e2a3a177c7" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46079 } }
I20260809 08:52:41.740456 23149 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e67e0c196334f3fb9ba76e2a3a177c7. Latest consensus state: current_term: 2 leader_uuid: "3e67e0c196334f3fb9ba76e2a3a177c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e67e0c196334f3fb9ba76e2a3a177c7" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46079 } } }
I20260809 08:52:41.740567 23149 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7 [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:41.740471 23151 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3e67e0c196334f3fb9ba76e2a3a177c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e67e0c196334f3fb9ba76e2a3a177c7" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46079 } } }
I20260809 08:52:41.740697 23151 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7 [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:41.742240 23176 catalog_manager.cc:1259] Loaded cluster ID: 8e3795f4a6494cfe9164586fae9f17f3
I20260809 08:52:41.742292 23176 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7: loading cluster ID for follower catalog manager: success
I20260809 08:52:41.743521 23176 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7: acquiring CA information for follower catalog manager: success
I20260809 08:52:41.743842 23176 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 08:52:41.743919 23168 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:52:41.744098 23168 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:52:41.744163 23168 catalog_manager.cc:1259] Loaded cluster ID: 8e3795f4a6494cfe9164586fae9f17f3
I20260809 08:52:41.744189 23168 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:52:41.744377 23168 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:52:41.744457 23168 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3e67e0c196334f3fb9ba76e2a3a177c7: Loaded TSK: 0
I20260809 08:52:41.744585 23168 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 08:52:41.809785 23146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:41.810000 23146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:41.810025 23146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:41.811211 23146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:41.811255 23146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:41.811282 23146 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.1
I20260809 08:52:41.812800 23146 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:34953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.1.keytab
--principal=oryx/127.8.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.1:40727
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.154.1
--webserver_port=33467
--tserver_master_addrs=127.8.154.62:46079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.1
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:41.813071 23146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:41.813329 23146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:41.819419 23182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:41.819557 23185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:41.819478 23183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:41.819705 23146 server_base.cc:1034] running on GCE node
I20260809 08:52:41.819926 23146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:41.820246 23146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:41.821403 23146 hybrid_clock.cc:648] HybridClock initialized: now 1786265561821392 us; error 28 us; skew 500 ppm
Aug 09 08:52:41 dist-test-slave-t3q3 krb5kdc[22275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265561, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:41.824250 23146 init.cc:377] Logged in from keytab as oryx/127.8.154.1@KRBTEST.COM (short username oryx)
I20260809 08:52:41.824679 23146 webserver.cc:460] Webserver started at http://127.8.154.1:33467/ using document root <none> and password file <none>
I20260809 08:52:41.824934 23146 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:41.825034 23146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:41.826468 23146 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:41.827144 23196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:41.827322 23146 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:41.827432 23146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-0/wal
uuid: "ab147b29f3be40329e7697140b6182d9"
format_stamp: "Formatted at 2026-08-09 08:52:41 on dist-test-slave-t3q3"
server_key: "ec1854de9b63720f228a847387869232"
server_key_iv: "e07309eb1d0d2a670c76d9b1ee90e5ba"
server_key_version: "encryptionkey@0"
I20260809 08:52:41.827788 23146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:41.848760 23146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:41.850982 23146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:41.851162 23146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:41.851418 23146 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:41.851735 23146 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:41.851790 23146 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:41.851826 23146 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:41.851871 23146 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:41.858628 23146 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.1:40727
I20260809 08:52:41.858685 23357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.1:40727 every 8 connection(s)
I20260809 08:52:41.859120 23146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
I20260809 08:52:41.860694 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 23146
I20260809 08:52:41.861002 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.2:34043
--local_ip_for_outbound_sockets=127.8.154.2
--tserver_master_addrs=127.8.154.62:46079
--webserver_port=35895
--webserver_interface=127.8.154.2
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:34953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.154.2
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 08:52:41 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:41 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:41.870353 23361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:41.859920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.1:46515)
0809 08:52:41.860148 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:41.860194 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:41.860819 (+ 625us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:41.861422 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:41.861425 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:41.862002 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:41.862198 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:41.862202 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:41.863977 (+ 1775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:41.863979 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:41.865345 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:41.865350 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:41.866308 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:41.866309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:41.866459 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:41.867115 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:41.867125 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:41.870089 (+ 2964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":93,"threads_started":1}
W20260809 08:52:41.870741 23364 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:41.860248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:46515 (local address 127.8.154.62:46079)
0809 08:52:41.860451 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:41.860460 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:41.860525 (+ 65us) server_negotiation.cc:407] Connection header received
0809 08:52:41.861034 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:41.861039 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:41.861125 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:41.861278 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:41.862350 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:41.863844 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:41.865461 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:41.866174 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:41.866317 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:41.870233 (+ 3916us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.1:46515: BlockingRecv error: recv got EOF from 127.8.154.1:46515 (error 108)
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":82,"threads_started":1}
W20260809 08:52:41.870819 23358 heartbeater.cc:644] Failed to heartbeat to 127.8.154.62:46079 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.154.62:46079: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Aug 09 08:52:41 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:41 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:41.876376 23361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:41.871097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.1:53375)
0809 08:52:41.871151 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:41.871163 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:41.871206 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:41.871530 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:41.871531 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:41.871701 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:41.871762 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:41.871764 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:41.872596 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:41.872598 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:41.873529 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:41.873532 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:41.873987 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:41.873988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:41.874058 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:41.874535 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:41.874540 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:41.876331 (+ 1791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 08:52:41.876495 23364 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:41.871155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:53375 (local address 127.8.154.62:46079)
0809 08:52:41.871395 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:41.871397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:41.871404 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:52:41.871427 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:41.871428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:41.871440 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:41.871497 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:41.871856 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:41.872506 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:41.873628 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:41.873903 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:41.873977 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:41.876432 (+ 2455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.1:53375: BlockingRecv error: recv got EOF from 127.8.154.1:53375 (error 108)
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":134,"threads_started":1}
Aug 09 08:52:41 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:41 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:41.880900 23361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:41.876634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.1:56275)
0809 08:52:41.876687 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:41.876697 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:41.876737 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:41.876895 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:41.876896 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:41.877000 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:41.877026 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:41.877027 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:41.877891 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:41.877891 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:41.878361 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:41.878362 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:41.878795 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:41.878796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:41.878832 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:41.879243 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:41.879248 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:41.880858 (+ 1610us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 08:52:41.881014 23364 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:41.876723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:56275 (local address 127.8.154.62:46079)
0809 08:52:41.876778 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:41.876780 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:41.876790 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:52:41.876812 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:41.876813 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:41.876823 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:41.876883 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:41.877089 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:41.877814 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:41.878434 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:41.878726 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:41.878790 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:41.880957 (+ 2167us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.1:56275: BlockingRecv error: recv got EOF from 127.8.154.1:56275 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260809 08:52:41.881343 23358 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 08:52:41.948767 23366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:41.948966 23366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:41.948988 23366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:41.950075 23366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:41.950110 23366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:41.950136 23366 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.2
I20260809 08:52:41.951500 23366 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:34953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.2.keytab
--principal=oryx/127.8.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.2:34043
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.154.2
--webserver_port=35895
--tserver_master_addrs=127.8.154.62:46079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.2
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:41.951684 23366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:41.951857 23366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:41.956888 23378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:41.956840 23380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:41.956845 23376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:41.957137 23366 server_base.cc:1034] running on GCE node
I20260809 08:52:41.957394 23366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:41.957633 23366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:41.958775 23366 hybrid_clock.cc:648] HybridClock initialized: now 1786265561958765 us; error 31 us; skew 500 ppm
Aug 09 08:52:41 dist-test-slave-t3q3 krb5kdc[22275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265561, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:41.961354 23366 init.cc:377] Logged in from keytab as oryx/127.8.154.2@KRBTEST.COM (short username oryx)
I20260809 08:52:41.961762 23366 webserver.cc:460] Webserver started at http://127.8.154.2:35895/ using document root <none> and password file <none>
I20260809 08:52:41.962024 23366 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:41.962116 23366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:41.963413 23366 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:41.963968 23387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:41.964159 23366 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:52:41.964268 23366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-1/wal
uuid: "5b4c353ca6824b5bae15f06efa77d53c"
format_stamp: "Formatted at 2026-08-09 08:52:41 on dist-test-slave-t3q3"
server_key: "e260277f9944c52b2aecc1aa7288063e"
server_key_iv: "69a519358a0e60ca36e1ba9890c052e1"
server_key_version: "encryptionkey@0"
I20260809 08:52:41.964591 23366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:41.981274 23366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:41.983126 23366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:41.983291 23366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:41.983527 23366 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:41.983846 23366 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:41.983909 23366 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:41.983950 23366 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:41.983978 23366 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:41.990083 23366 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.2:34043
I20260809 08:52:41.990123 23556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.2:34043 every 8 connection(s)
I20260809 08:52:41.990475 23366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
Aug 09 08:52:41 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:41 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:41.997391 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 23366
I20260809 08:52:41.997751 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.3:42813
--local_ip_for_outbound_sockets=127.8.154.3
--tserver_master_addrs=127.8.154.62:46079
--webserver_port=33371
--webserver_interface=127.8.154.3
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:34953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.8.154.3
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:41.997929 23364 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:41.991185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:37285 (local address 127.8.154.62:46079)
0809 08:52:41.991254 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:41.991256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:41.991481 (+ 225us) server_negotiation.cc:407] Connection header received
0809 08:52:41.991973 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:41.991975 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:41.991990 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:41.992044 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:41.993029 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:41.993521 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:41.994480 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:41.994663 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:41.994713 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:41.997842 (+ 3129us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.2:37285: BlockingRecv error: recv got EOF from 127.8.154.2:37285 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 08:52:41.997946 23560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:41.991168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.2:37285)
0809 08:52:41.991324 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:41.991360 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:41.991883 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:41.992159 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:41.992163 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:41.992742 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:41.992940 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:41.992944 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:41.993602 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:41.993602 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:41.994407 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:41.994409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:41.994736 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:41.994737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:41.994834 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:41.995408 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:41.995416 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:41.997730 (+ 2314us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":91,"threads_started":1}
W20260809 08:52:41.998179 23558 heartbeater.cc:644] Failed to heartbeat to 127.8.154.62:46079 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.154.62:46079: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Aug 09 08:52:41 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:41 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:42.002897 23560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:41.998436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.2:44845)
0809 08:52:41.998478 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:41.998488 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:41.998522 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:41.998633 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:41.998634 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:41.998790 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:41.998851 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:41.998852 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:41.999479 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:41.999480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:42.000167 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:42.000169 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:42.000535 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:42.000536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:42.000601 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:42.000997 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:42.001003 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:42.002813 (+ 1810us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 08:52:42.002948 23364 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:41.998492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:44845 (local address 127.8.154.62:46079)
0809 08:52:41.998530 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:41.998532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:41.998542 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:52:41.998572 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:41.998573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:41.998580 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:41.998626 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:41.998940 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:41.999389 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:42.000265 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:42.000450 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:42.000534 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:42.002884 (+ 2350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.2:44845: BlockingRecv error: recv got EOF from 127.8.154.2:44845 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
Aug 09 08:52:42 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:42 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:42.007300 23560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:42.003132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.2:49191)
0809 08:52:42.003272 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:42.003281 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:42.003354 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:42.003491 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:42.003491 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:42.003615 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:42.003641 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:42.003642 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:42.004296 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:42.004297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:42.004802 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:42.004803 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:42.005186 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:42.005186 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:42.005243 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:42.005677 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:42.005681 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:42.007220 (+ 1539us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":97,"threads_started":1}
W20260809 08:52:42.007350 23364 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:42.003210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:49191 (local address 127.8.154.62:46079)
0809 08:52:42.003263 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:42.003264 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:42.003339 (+ 75us) server_negotiation.cc:407] Connection header received
0809 08:52:42.003421 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:42.003422 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:42.003431 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:42.003487 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:42.003715 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:42.004218 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:42.004901 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:42.005099 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:42.005184 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:42.007288 (+ 2104us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.2:49191: BlockingRecv error: recv got EOF from 127.8.154.2:49191 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260809 08:52:42.007416 23558 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 08:52:42.082598 23561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:42.082825 23561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:42.082866 23561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:42.084103 23561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:42.084167 23561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:42.084218 23561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.3
I20260809 08:52:42.085639 23561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:34953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/krb5kdc/oryx_127.8.154.3.keytab
--principal=oryx/127.8.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.3:42813
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.154.3
--webserver_port=33371
--tserver_master_addrs=127.8.154.62:46079
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.3
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:42.085922 23561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:42.086145 23561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:42.091310 23571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:42.091327 23575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:42.091306 23561 server_base.cc:1034] running on GCE node
W20260809 08:52:42.091327 23570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:42.091832 23561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:42.092021 23561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:42.093201 23561 hybrid_clock.cc:648] HybridClock initialized: now 1786265562093195 us; error 37 us; skew 500 ppm
Aug 09 08:52:42 dist-test-slave-t3q3 krb5kdc[22275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265562, etypes {rep=17 tkt=17 ses=17}, oryx/127.8.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:42.095857 23561 init.cc:377] Logged in from keytab as oryx/127.8.154.3@KRBTEST.COM (short username oryx)
I20260809 08:52:42.096372 23561 webserver.cc:460] Webserver started at http://127.8.154.3:33371/ using document root <none> and password file <none>
I20260809 08:52:42.096598 23561 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:42.096688 23561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:42.098165 23561 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:42.098906 23585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:42.099174 23561 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:42.099311 23561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-2/wal
uuid: "4dcd06121c104956992a40aa6b9754cd"
format_stamp: "Formatted at 2026-08-09 08:52:41 on dist-test-slave-t3q3"
server_key: "50d0c0e76fe27c5a047912f3bda027bf"
server_key_iv: "25eaa8036587060b74df7a524274c06d"
server_key_version: "encryptionkey@0"
I20260809 08:52:42.099653 23561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:42.130129 23561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:42.132647 23561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:42.132817 23561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:42.133076 23561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:42.133399 23561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:42.133455 23561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:42.133494 23561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:42.133541 23561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:42.139514 23561 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.3:42813
I20260809 08:52:42.139544 23749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.3:42813 every 8 connection(s)
I20260809 08:52:42.139937 23561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
I20260809 08:52:42.144344 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 23561
Aug 09 08:52:42 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.3@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:42 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.3@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
W20260809 08:52:42.147987 23364 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:42.140758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:59985 (local address 127.8.154.62:46079)
0809 08:52:42.140809 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:42.140812 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:42.141124 (+ 312us) server_negotiation.cc:407] Connection header received
0809 08:52:42.141726 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:42.141727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:42.141738 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:42.141794 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:42.142762 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:42.143243 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:42.144241 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:42.144542 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:42.144618 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:42.147929 (+ 3311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.3:59985: BlockingRecv error: recv got EOF from 127.8.154.3:59985 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:52:42.148061 23753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:42.140766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.3:59985)
0809 08:52:42.140952 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:42.141009 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:42.141622 (+ 613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:42.141881 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:42.141884 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:42.142483 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:42.142682 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:42.142686 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:42.143322 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:42.143323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:42.144149 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:42.144151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:42.144660 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:42.144661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:42.144816 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:42.145391 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:42.145399 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:42.147863 (+ 2464us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":92,"threads_started":1}
W20260809 08:52:42.148447 23750 heartbeater.cc:644] Failed to heartbeat to 127.8.154.62:46079 (0 consecutive failures): Not authorized: Failed to ping master at 127.8.154.62:46079: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Aug 09 08:52:42 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.3@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:42 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.3@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:42.153075 23367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:42.144909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55448 (local address 127.8.154.62:46079)
0809 08:52:42.144960 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:42.144962 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:42.145085 (+ 123us) server_negotiation.cc:407] Connection header received
0809 08:52:42.145154 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:42.145155 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:42.145167 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:42.145268 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:42.145895 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:42.146527 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:42.147696 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:42.147891 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:42.147972 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:42.149490 (+ 1518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:42.149495 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:42.149495 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:42.149506 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:42.151356 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:42.151680 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:42.151684 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:42.151685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:42.151802 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:42.152206 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:42.152207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:42.152207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:42.152454 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:42.152592 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:42.152833 (+ 241us) server_negotiation.cc:299] Negotiation successful
0809 08:52:42.152957 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 08:52:42.153513 23753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:42.148695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.3:37589)
0809 08:52:42.148750 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:42.148760 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:42.148796 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:42.148937 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:42.148938 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:42.149088 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:42.149149 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:42.149151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:42.149898 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:42.149899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:42.150602 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:42.150604 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:42.151038 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:42.151039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:42.151089 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:42.151488 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:42.151493 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:42.153454 (+ 1961us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 08:52:42.153615 23364 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:42.148743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:37589 (local address 127.8.154.62:46079)
0809 08:52:42.148777 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:42.148779 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:42.148787 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:52:42.148856 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:42.148856 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:42.148863 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:42.148915 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:42.149228 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:42.149809 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:42.150679 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:42.150959 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:42.151033 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:42.153539 (+ 2506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.3:37589: BlockingRecv error: recv got EOF from 127.8.154.3:37589 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
Aug 09 08:52:42 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.3@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:42 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.3@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:42.157508 23753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:42.153814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.3:49369)
0809 08:52:42.153854 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:42.153865 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:42.153904 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:42.154090 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:42.154091 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:42.154227 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:42.154253 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:42.154254 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:42.154830 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:42.154831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:42.155352 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:42.155354 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:42.155648 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:42.155648 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:42.155684 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:42.155982 (+ 298us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:42.155986 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:42.157468 (+ 1482us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":21}
W20260809 08:52:42.157639 23364 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:42.153904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:49369 (local address 127.8.154.62:46079)
0809 08:52:42.153965 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:42.153966 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:42.153975 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:52:42.153995 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:42.153996 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:42.154006 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:42.154063 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:42.154316 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:42.154742 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:42.155419 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:42.155588 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:42.155635 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:42.157566 (+ 1931us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.3:49369: BlockingRecv error: recv got EOF from 127.8.154.3:49369 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260809 08:52:42.157655 23750 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 08:52:42 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:42 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:42.886263 23759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:42.881728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.1:46713)
0809 08:52:42.881892 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:42.881904 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:42.881957 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:42.882088 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:42.882089 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:42.882285 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:42.882339 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:42.882342 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:42.882974 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:42.882974 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:42.883528 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:42.883530 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:42.883861 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:42.883861 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:42.883905 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:42.884374 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:42.884379 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:42.886188 (+ 1809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":93,"threads_started":1}
W20260809 08:52:42.886361 23760 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:42.881806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:46713 (local address 127.8.154.62:46079)
0809 08:52:42.881935 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:42.881937 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:42.881947 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:52:42.882004 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:42.882004 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:42.882014 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:42.882073 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:42.882409 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:42.882906 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:42.883601 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:42.883795 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:42.883845 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:42.886289 (+ 2444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.1:46713: BlockingRecv error: recv got EOF from 127.8.154.1:46713 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":66,"threads_started":1}
Aug 09 08:52:43 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:43 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:43.012542 23761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:43.007886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.2:38219)
0809 08:52:43.008182 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:43.008195 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:43.008268 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:43.008474 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:43.008475 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:43.008714 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:43.008766 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:43.008768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:43.009400 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:43.009401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:43.009951 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:43.009952 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:43.010269 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:43.010270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:43.010312 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:43.010733 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:43.010738 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:43.012474 (+ 1736us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":205,"threads_started":1}
W20260809 08:52:43.012609 23760 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:43.008098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:38219 (local address 127.8.154.62:46079)
0809 08:52:43.008149 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:43.008151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:43.008274 (+ 123us) server_negotiation.cc:407] Connection header received
0809 08:52:43.008361 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:43.008362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:43.008379 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:43.008451 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:43.008835 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:43.009331 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:43.010023 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:43.010207 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:43.010256 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:43.012563 (+ 2307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.2:38219: BlockingRecv error: recv got EOF from 127.8.154.2:38219 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 09 08:52:43 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.3@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:43 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.3@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:43.162858 23762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:43.158163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.3:43853)
0809 08:52:43.158311 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:43.158323 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:43.158387 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:43.158561 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:43.158562 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:43.158732 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:43.158788 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:43.158790 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:43.159472 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:43.159473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:43.160054 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:43.160080 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:43.160398 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:43.160399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:43.160442 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:43.160921 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:43.160926 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:43.162790 (+ 1864us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":96,"threads_started":1}
W20260809 08:52:43.162940 23760 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:43.158214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:43853 (local address 127.8.154.62:46079)
0809 08:52:43.158263 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:43.158265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:43.158366 (+ 101us) server_negotiation.cc:407] Connection header received
0809 08:52:43.158470 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:43.158471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:43.158484 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:43.158533 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:43.158882 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:43.159388 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:43.160149 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:43.160333 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:43.160388 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:43.162891 (+ 2503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.3:43853: BlockingRecv error: recv got EOF from 127.8.154.3:43853 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 09 08:52:43 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:43 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:43.891722 23763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:43.886867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.1:53513)
0809 08:52:43.887011 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:43.887024 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:43.887093 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:43.887305 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:43.887306 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:43.887533 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:43.887586 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:43.887588 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:43.888391 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:43.888392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:43.888934 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:43.888936 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:43.889319 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:43.889320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:43.889388 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:43.889841 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:43.889846 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:43.891654 (+ 1808us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":89,"threads_started":1}
W20260809 08:52:43.891872 23764 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:43.887009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:53513 (local address 127.8.154.62:46079)
0809 08:52:43.887153 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:43.887155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:43.887165 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:52:43.887203 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:43.887204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:43.887215 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:43.887290 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:43.887685 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:43.888296 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:43.889030 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:43.889240 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:43.889320 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:43.891777 (+ 2457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.1:53513: BlockingRecv error: recv got EOF from 127.8.154.1:53513 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":78,"threads_started":1}
Aug 09 08:52:44 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:44 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:44.018142 23766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:44.013124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.2:52375)
0809 08:52:44.013372 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:44.013388 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:44.013496 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:44.013665 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:44.013666 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:44.013864 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:44.013915 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:44.013916 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:44.014612 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:44.014613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:44.015148 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:44.015149 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:44.015559 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:44.015560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:44.015621 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:44.016161 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:44.016167 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:44.018075 (+ 1908us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":98,"threads_started":1}
W20260809 08:52:44.018244 23764 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:44.013266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:52375 (local address 127.8.154.62:46079)
0809 08:52:44.013377 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:44.013380 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:44.013474 (+ 94us) server_negotiation.cc:407] Connection header received
0809 08:52:44.013564 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:44.013566 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:44.013580 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:44.013655 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:44.013998 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:44.014516 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:44.015288 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:44.015482 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:44.015561 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:44.018193 (+ 2632us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.2:52375: BlockingRecv error: recv got EOF from 127.8.154.2:52375 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 09 08:52:44 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.3@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:44 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.3@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:44.168033 23768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:44.163359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.3:37037)
0809 08:52:44.163523 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:44.163537 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:44.163603 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:44.163748 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:44.163749 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:44.163922 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:44.163975 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:44.163977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:44.164724 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:44.164725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:44.165277 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:44.165278 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:44.165594 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:44.165595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:44.165639 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:44.166061 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:44.166065 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:44.167967 (+ 1902us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":76,"threads_started":1}
W20260809 08:52:44.168164 23764 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:44.163501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:37037 (local address 127.8.154.62:46079)
0809 08:52:44.163570 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:44.163572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:44.163584 (+ 12us) server_negotiation.cc:407] Connection header received
0809 08:52:44.163663 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:44.163664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:44.163676 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:44.163730 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:44.164040 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:44.164647 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:44.165349 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:44.165530 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:44.165582 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:44.168079 (+ 2497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.3:37037: BlockingRecv error: recv got EOF from 127.8.154.3:37037 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 09 08:52:44 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:44 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:44.897891 23769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:44.892319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.1:40135)
0809 08:52:44.892495 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:44.892520 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:44.892580 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:44.892766 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:44.892768 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:44.893060 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:44.893122 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:44.893124 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:44.893945 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:44.893946 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:44.894911 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:44.894915 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:44.895385 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:44.895386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:44.895433 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:44.895883 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:44.895888 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:44.897816 (+ 1928us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":87,"threads_started":1}
W20260809 08:52:44.898020 23770 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:44.892385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:40135 (local address 127.8.154.62:46079)
0809 08:52:44.892525 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:44.892578 (+ 53us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:44.892589 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:52:44.892659 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:44.892660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:44.892674 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:44.892740 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:44.893268 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:44.893826 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:44.895034 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:44.895315 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:44.895375 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:44.897928 (+ 2553us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.1:40135: BlockingRecv error: recv got EOF from 127.8.154.1:40135 (error 108)
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":77,"threads_started":1}
Aug 09 08:52:45 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:45 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:45.023641 23771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:45.018794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.2:46667)
0809 08:52:45.018956 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:45.018975 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:45.019034 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:45.019169 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:45.019170 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:45.019344 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:45.019398 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:45.019399 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:45.020156 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:45.020156 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:45.020712 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:45.020714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:45.021067 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:45.021068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:45.021112 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:45.021562 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:45.021566 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:45.023506 (+ 1940us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":108,"threads_started":1}
W20260809 08:52:45.023708 23770 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:45.018897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:46667 (local address 127.8.154.62:46079)
0809 08:52:45.018952 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:45.018954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:45.019024 (+ 70us) server_negotiation.cc:407] Connection header received
0809 08:52:45.019083 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:45.019083 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:45.019094 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:45.019149 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:45.019482 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:45.020041 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:45.020793 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:45.020987 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:45.021047 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:45.023631 (+ 2584us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.2:46667: BlockingRecv error: recv got EOF from 127.8.154.2:46667 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 09 08:52:45 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.3@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:45 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.3@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:45.173324 23774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:45.168622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.3:32987)
0809 08:52:45.168793 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:45.168804 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:45.168876 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:45.169028 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:45.169029 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:45.169198 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:45.169253 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:45.169254 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:45.169919 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:45.169919 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:45.170460 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:45.170461 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:45.170832 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:45.170833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:45.170877 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:45.171341 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:45.171346 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:45.173247 (+ 1901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":95,"threads_started":1}
W20260809 08:52:45.173448 23770 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:45.168677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:32987 (local address 127.8.154.62:46079)
0809 08:52:45.168728 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:45.168730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:45.168903 (+ 173us) server_negotiation.cc:407] Connection header received
0809 08:52:45.168934 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:45.168935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:45.168949 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:45.169010 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:45.169323 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:45.169849 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:45.170557 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:45.170757 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:45.170821 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:45.173347 (+ 2526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.3:32987: BlockingRecv error: recv got EOF from 127.8.154.3:32987 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 09 08:52:45 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:45 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:45.904225 23775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:45.898511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.1:51615)
0809 08:52:45.898681 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:45.898694 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:45.898821 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:45.899049 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:45.899050 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:45.899221 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:45.899274 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:45.899275 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:45.900200 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:45.900202 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:45.901027 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:45.901030 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:45.901407 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:45.901409 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:45.901469 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:45.901984 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:45.901990 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:45.904132 (+ 2142us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":97,"threads_started":1}
W20260809 08:52:45.904266 23776 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:45.898554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:51615 (local address 127.8.154.62:46079)
0809 08:52:45.898691 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:45.898693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:45.898796 (+ 103us) server_negotiation.cc:407] Connection header received
0809 08:52:45.898927 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:45.898928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:45.898947 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:45.899008 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:45.899426 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:45.900078 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:45.901129 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:45.901330 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:45.901383 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:45.904199 (+ 2816us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.1:51615: BlockingRecv error: recv got EOF from 127.8.154.1:51615 (error 108)
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":82,"threads_started":1}
Aug 09 08:52:46 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:46 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:46.032450 23778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:46.024242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.2:48455)
0809 08:52:46.024432 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:46.024443 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:46.024520 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:46.024667 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:46.024668 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:46.024871 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:46.024924 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:46.024926 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:46.025633 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:46.025634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:46.026187 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:46.026189 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:46.026626 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:46.026627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:46.026693 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:46.030373 (+ 3680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:46.030378 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:46.032388 (+ 2010us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":128,"threads_started":1}
W20260809 08:52:46.032582 23776 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:46.024286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:48455 (local address 127.8.154.62:46079)
0809 08:52:46.024359 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:46.024361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:46.024524 (+ 163us) server_negotiation.cc:407] Connection header received
0809 08:52:46.024588 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:46.024589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:46.024603 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:46.024656 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:46.025015 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:46.025540 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:46.026288 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:46.026545 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:46.026628 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:46.032498 (+ 5870us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.2:48455: BlockingRecv error: recv got EOF from 127.8.154.2:48455 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 09 08:52:46 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.3@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:46 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.3@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:46.185482 23779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:46.173876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.3:49331)
0809 08:52:46.174068 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:46.174080 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:46.174144 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:46.174292 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:46.174293 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:46.174472 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:46.174526 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:46.174528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:46.175242 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:46.175243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:46.175993 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:46.175997 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:46.176451 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:46.176452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:46.176499 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:46.183502 (+ 7003us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:46.183507 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:46.185421 (+ 1914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":126,"threads_started":1}
W20260809 08:52:46.185642 23776 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:46.173973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:49331 (local address 127.8.154.62:46079)
0809 08:52:46.174064 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:46.174066 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:46.174141 (+ 75us) server_negotiation.cc:407] Connection header received
0809 08:52:46.174191 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:46.174192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:46.174207 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:46.174271 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:46.174633 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:46.175157 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:46.176099 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:46.176379 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:46.176435 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:46.185563 (+ 9128us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.3:49331: BlockingRecv error: recv got EOF from 127.8.154.3:49331 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 09 08:52:46 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:46 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:46.909524 23780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:46.904722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.1:53807)
0809 08:52:46.904856 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:46.904867 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:46.904929 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:46.905095 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:46.905096 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:46.905270 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:46.905322 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:46.905323 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:46.906134 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:46.906135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:46.906660 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:46.906661 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:46.907046 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:46.907047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:46.907092 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:46.907605 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:46.907610 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:46.909464 (+ 1854us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":73,"threads_started":1}
W20260809 08:52:46.909720 23781 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:46.904777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:53807 (local address 127.8.154.62:46079)
0809 08:52:46.904920 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:46.904922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:46.904933 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:52:46.904984 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:46.904985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:46.904995 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:46.905076 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:46.905416 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:46.906053 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:46.906751 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:46.906970 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:46.907039 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:46.909620 (+ 2581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.1:53807: BlockingRecv error: recv got EOF from 127.8.154.1:53807 (error 108)
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":98,"threads_started":1}
Aug 09 08:52:47 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:47 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:47.037945 23782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:47.032983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.2:36683)
0809 08:52:47.033173 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:47.033184 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:47.033248 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:47.033398 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:47.033399 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:47.033587 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:47.033643 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:47.033644 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:47.034471 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:47.034472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:47.035095 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:47.035097 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:47.035467 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:47.035469 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:47.035526 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:47.036111 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:47.036117 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:47.037884 (+ 1767us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":103,"threads_started":1}
W20260809 08:52:47.038120 23781 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:47.033120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:36683 (local address 127.8.154.62:46079)
0809 08:52:47.033179 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:47.033181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:47.033243 (+ 62us) server_negotiation.cc:407] Connection header received
0809 08:52:47.033304 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:47.033304 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:47.033316 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:47.033373 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:47.033711 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:47.034372 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:47.035171 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:47.035364 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:47.035419 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:47.038005 (+ 2586us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.2:36683: BlockingRecv error: recv got EOF from 127.8.154.2:36683 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 09 08:52:47 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.3@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:47 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.3@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:47.190932 23783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:47.186032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.3:53955)
0809 08:52:47.186159 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:47.186173 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:47.186252 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:47.186470 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:47.186472 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:47.186738 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:47.186807 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:47.186809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:47.187515 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:47.187516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:47.188084 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:47.188086 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:47.188469 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:47.188470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:47.188536 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:47.189019 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:47.189023 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:47.190868 (+ 1845us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":71,"threads_started":1}
W20260809 08:52:47.191062 23781 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:47.186079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:53955 (local address 127.8.154.62:46079)
0809 08:52:47.186157 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:47.186159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:47.186251 (+ 92us) server_negotiation.cc:407] Connection header received
0809 08:52:47.186359 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:47.186360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:47.186377 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:47.186444 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:47.186928 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:47.187440 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:47.188177 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:47.188390 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:47.188470 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:47.190960 (+ 2490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.3:53955: BlockingRecv error: recv got EOF from 127.8.154.3:53955 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 09 08:52:47 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:47 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:47.915341 23784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:47.910016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.1:44637)
0809 08:52:47.910194 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:47.910205 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:47.910272 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:47.910441 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:47.910442 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:47.910614 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:47.910667 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:47.910669 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:47.911400 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:47.911401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:47.912210 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:47.912214 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:47.912649 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:47.912650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:47.912696 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:47.913270 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:47.913275 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:47.915283 (+ 2008us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":106,"threads_started":1}
W20260809 08:52:47.915499 23785 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:47.910136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:44637 (local address 127.8.154.62:46079)
0809 08:52:47.910309 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:47.910311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:47.910321 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:52:47.910348 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:47.910348 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:47.910359 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:47.910427 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:47.910746 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:47.911326 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:47.912307 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:47.912579 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:47.912647 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:47.915391 (+ 2744us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.1:44637: BlockingRecv error: recv got EOF from 127.8.154.1:44637 (error 108)
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":108,"threads_started":1}
Aug 09 08:52:48 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:48 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:48.043537 23787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:48.038463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.2:59765)
0809 08:52:48.038615 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:48.038631 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:48.038686 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:48.038818 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:48.038819 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:48.038996 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:48.039051 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:48.039052 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:48.039695 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:48.039700 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:48.040353 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:48.040355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:48.040726 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:48.040757 (+ 31us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:48.040814 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:48.041342 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:48.041347 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:48.043479 (+ 2132us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":96,"threads_started":1}
W20260809 08:52:48.043691 23785 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:48.038546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:59765 (local address 127.8.154.62:46079)
0809 08:52:48.038616 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:48.038618 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:48.038685 (+ 67us) server_negotiation.cc:407] Connection header received
0809 08:52:48.038733 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:48.038734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:48.038745 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:48.038798 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:48.039114 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:48.039614 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:48.040424 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:48.040652 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:48.040709 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:48.043591 (+ 2882us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.2:59765: BlockingRecv error: recv got EOF from 127.8.154.2:59765 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 09 08:52:48 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.3@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:48 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.3@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:48.196650 23788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:48.191520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.3:38457)
0809 08:52:48.191743 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:48.191757 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:48.191861 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:48.192035 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:48.192036 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:48.192268 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:48.192321 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:48.192323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:48.193039 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:48.193040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:48.193619 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:48.193643 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:48.194035 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:48.194036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:48.194105 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:48.194666 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:48.194671 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:48.196552 (+ 1881us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":118,"threads_started":1}
W20260809 08:52:48.196725 23785 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:48.191632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:38457 (local address 127.8.154.62:46079)
0809 08:52:48.191732 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:48.191735 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:48.191841 (+ 106us) server_negotiation.cc:407] Connection header received
0809 08:52:48.191934 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:48.191935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:48.191956 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:48.192027 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:48.192413 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:48.192929 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:48.193747 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:48.193946 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:48.194023 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:48.196645 (+ 2622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.3:38457: BlockingRecv error: recv got EOF from 127.8.154.3:38457 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 09 08:52:48 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:48 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:48.920819 23790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:48.915899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.1:56431)
0809 08:52:48.916056 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:48.916115 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:48.916185 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:48.916345 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:48.916346 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:48.916523 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:48.916593 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:48.916595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:48.917329 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:48.917330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:48.917895 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:48.917897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:48.918240 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:48.918241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:48.918288 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:48.918759 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:48.918765 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:48.920749 (+ 1984us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":102,"threads_started":1}
W20260809 08:52:48.920936 23791 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:48.915989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:56431 (local address 127.8.154.62:46079)
0809 08:52:48.916126 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:48.916133 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:48.916167 (+ 34us) server_negotiation.cc:407] Connection header received
0809 08:52:48.916246 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:48.916247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:48.916258 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:48.916324 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:48.916661 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:48.917226 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:48.917975 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:48.918171 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:48.918228 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:48.920848 (+ 2620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.1:56431: BlockingRecv error: recv got EOF from 127.8.154.1:56431 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":100,"threads_started":1}
Aug 09 08:52:49 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:49 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:49.048834 23792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:49.044016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.2:46883)
0809 08:52:49.044170 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:49.044180 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:49.044244 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:49.044363 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:49.044364 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:49.044538 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:49.044591 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:49.044593 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:49.045241 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:49.045242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:49.045785 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:49.045787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:49.046174 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:49.046174 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:49.046217 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:49.046699 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:49.046704 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:49.048774 (+ 2070us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":92,"threads_started":1}
W20260809 08:52:49.048991 23791 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:49.044106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:46883 (local address 127.8.154.62:46079)
0809 08:52:49.044180 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:49.044182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:49.044242 (+ 60us) server_negotiation.cc:407] Connection header received
0809 08:52:49.044292 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:49.044293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:49.044304 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:49.044358 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:49.044650 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:49.045159 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:49.045889 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:49.046096 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:49.046163 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:49.048907 (+ 2744us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.2:46883: BlockingRecv error: recv got EOF from 127.8.154.2:46883 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 09 08:52:49 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.3@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:49 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.3@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:49.202310 23793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:49.197270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.3:44801)
0809 08:52:49.197433 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:49.197443 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:49.197500 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:49.197614 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:49.197614 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:49.197807 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:49.197867 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:49.197878 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:49.198628 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:49.198630 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:49.199234 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:49.199236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:49.199605 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:49.199606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:49.199697 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:49.200272 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:49.200277 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:49.202240 (+ 1963us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":107,"threads_started":1}
W20260809 08:52:49.202432 23791 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:49.197336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:44801 (local address 127.8.154.62:46079)
0809 08:52:49.197419 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:49.197421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:49.197508 (+ 87us) server_negotiation.cc:407] Connection header received
0809 08:52:49.197553 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:49.197554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:49.197567 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:49.197609 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:49.197949 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:49.198539 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:49.199314 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:49.199517 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:49.199601 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:49.202335 (+ 2734us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.3:44801: BlockingRecv error: recv got EOF from 127.8.154.3:44801 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 08:52:49 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:49 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:49.926013 23794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:49.921328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.1:53609)
0809 08:52:49.921465 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:49.921483 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:49.921532 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:49.921757 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:49.921758 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:49.921932 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:49.921984 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:49.921986 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:49.922640 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:49.922640 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:49.923174 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:49.923176 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:49.923529 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:49.923530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:49.923572 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:49.924013 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:49.924018 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:49.925914 (+ 1896us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":77,"threads_started":1}
W20260809 08:52:49.926092 23795 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:49.921480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:53609 (local address 127.8.154.62:46079)
0809 08:52:49.921623 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:49.921625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:49.921633 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:52:49.921662 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:49.921663 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:49.921672 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:49.921737 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:49.922068 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:49.922568 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:49.923258 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:49.923458 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:49.923519 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:49.925992 (+ 2473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.1:53609: BlockingRecv error: recv got EOF from 127.8.154.1:53609 (error 108)
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":89,"threads_started":1}
Aug 09 08:52:50 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:50 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:50.054253 23796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:50.049314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.2:51025)
0809 08:52:50.049466 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:50.049476 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:50.049560 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:50.049696 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:50.049697 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:50.049884 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:50.049933 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:50.049935 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:50.050766 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:50.050767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:50.051317 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:50.051319 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:50.051695 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:50.051696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:50.051756 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:50.052265 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:50.052269 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:50.054151 (+ 1882us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":67,"threads_started":1}
W20260809 08:52:50.054308 23795 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:50.049364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:51025 (local address 127.8.154.62:46079)
0809 08:52:50.049412 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:50.049414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:50.049543 (+ 129us) server_negotiation.cc:407] Connection header received
0809 08:52:50.049619 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:50.049620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:50.049634 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:50.049680 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:50.050014 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:50.050665 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:50.051409 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:50.051617 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:50.051695 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:50.054242 (+ 2547us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.2:51025: BlockingRecv error: recv got EOF from 127.8.154.2:51025 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 09 08:52:50 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.3@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:50 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.3@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:50.207739 23798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:50.202797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.3:57155)
0809 08:52:50.203004 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:50.203019 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:50.203099 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:50.203267 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:50.203268 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:50.203462 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:50.203511 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:50.203513 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:50.204294 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:50.204296 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:50.204841 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:50.204843 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:50.205259 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:50.205260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:50.205329 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:50.205806 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:50.205810 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:50.207653 (+ 1843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":92,"threads_started":1}
W20260809 08:52:50.207829 23795 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:50.202869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:57155 (local address 127.8.154.62:46079)
0809 08:52:50.202949 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:50.202951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:50.203084 (+ 133us) server_negotiation.cc:407] Connection header received
0809 08:52:50.203166 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:50.203167 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:50.203188 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:50.203257 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:50.203603 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:50.204207 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:50.204953 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:50.205173 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:50.205257 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:50.207763 (+ 2506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.3:57155: BlockingRecv error: recv got EOF from 127.8.154.3:57155 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 09 08:52:50 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:50 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:50.931888 23800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:50.926537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.1:36547)
0809 08:52:50.926691 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:50.926702 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:50.926757 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:50.926955 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:50.926956 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:50.927138 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:50.927194 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:50.927196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:50.927894 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:50.927895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:50.928457 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:50.928459 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:50.928797 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:50.928798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:50.928844 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:50.929386 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:50.929391 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:50.931788 (+ 2397us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":88,"threads_started":1}
W20260809 08:52:50.931946 23801 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:50.926608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:36547 (local address 127.8.154.62:46079)
0809 08:52:50.926736 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:50.926738 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:50.926748 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:52:50.926809 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:50.926810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:50.926827 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:50.926904 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:50.927270 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:50.927810 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:50.928530 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:50.928729 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:50.928782 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:50.931873 (+ 3091us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.1:36547: BlockingRecv error: recv got EOF from 127.8.154.1:36547 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":71,"threads_started":1}
Aug 09 08:52:51 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:51 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:51.063436 23802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:51.054781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.2:60455)
0809 08:52:51.055022 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:51.055033 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:51.055121 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:51.055310 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:51.055311 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:51.055497 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:51.055551 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:51.055553 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:51.056308 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:51.056309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:51.056857 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:51.056858 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:51.057245 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:51.057246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:51.057313 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:51.061287 (+ 3974us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:51.061291 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:51.063366 (+ 2075us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":91,"threads_started":1}
W20260809 08:52:51.063585 23801 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:51.054857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:60455 (local address 127.8.154.62:46079)
0809 08:52:51.054947 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:51.054950 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:51.055101 (+ 151us) server_negotiation.cc:407] Connection header received
0809 08:52:51.055220 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:51.055220 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:51.055232 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:51.055288 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:51.055637 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:51.056216 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:51.056946 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:51.057153 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:51.057243 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:51.063457 (+ 6214us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.2:60455: BlockingRecv error: recv got EOF from 127.8.154.2:60455 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 09 08:52:51 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.3@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:51 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.3@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:51.213204 23803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:51.208376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.3:33661)
0809 08:52:51.208525 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:51.208537 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:51.208609 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:51.208752 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:51.208753 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:51.208936 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:51.208991 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:51.208992 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:51.209693 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:51.209693 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:51.210247 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:51.210249 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:51.210598 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:51.210599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:51.210642 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:51.211152 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:51.211157 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:51.213146 (+ 1989us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":94,"threads_started":1}
W20260809 08:52:51.213425 23801 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:51.208441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:33661 (local address 127.8.154.62:46079)
0809 08:52:51.208520 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:51.208522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:51.208613 (+ 91us) server_negotiation.cc:407] Connection header received
0809 08:52:51.208653 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:51.208654 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:51.208667 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:51.208731 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:51.209078 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:51.209619 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:51.210329 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:51.210518 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:51.210584 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:51.213350 (+ 2766us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.3:33661: BlockingRecv error: recv got EOF from 127.8.154.3:33661 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 09 08:52:51 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:51 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:51.937111 23804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:51.932423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.1:46721)
0809 08:52:51.932583 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:51.932600 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:51.932663 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:51.932825 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:51.932826 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:51.933040 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:51.933094 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:51.933096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:51.933748 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:51.933749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:51.934336 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:51.934338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:51.934678 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:51.934679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:51.934727 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:51.935199 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:51.935204 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:51.937011 (+ 1807us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":90,"threads_started":1}
W20260809 08:52:51.937171 23805 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:51.932507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:46721 (local address 127.8.154.62:46079)
0809 08:52:51.932634 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:51.932636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:51.932646 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:52:51.932729 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:51.932729 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:51.932741 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:51.932802 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:51.933163 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:51.933678 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:51.934411 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:51.934607 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:51.934657 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:51.937097 (+ 2440us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.1:46721: BlockingRecv error: recv got EOF from 127.8.154.1:46721 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":55,"threads_started":1}
Aug 09 08:52:52 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:52 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:52.069046 23806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:52.063995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.2:40853)
0809 08:52:52.064195 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:52.064205 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:52.064274 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:52.064508 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:52.064509 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:52.064691 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:52.064746 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:52.064747 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:52.065384 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:52.065385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:52.065920 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:52.065922 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:52.066245 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:52.066246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:52.066287 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:52.066754 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:52.066758 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:52.068942 (+ 2184us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":117,"threads_started":1}
W20260809 08:52:52.069096 23805 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:52.064084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:40853 (local address 127.8.154.62:46079)
0809 08:52:52.064168 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:52.064171 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:52.064378 (+ 207us) server_negotiation.cc:407] Connection header received
0809 08:52:52.064413 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:52.064414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:52.064427 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:52.064491 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:52.064818 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:52.065318 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:52.065991 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:52.066178 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:52.066229 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:52.069031 (+ 2802us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.2:40853: BlockingRecv error: recv got EOF from 127.8.154.2:40853 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 09 08:52:52 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.3@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:52 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.3@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:52.218628 23807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:52.213686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.3:55095)
0809 08:52:52.213979 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:52.213998 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:52.214108 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:52.214248 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:52.214249 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:52.214443 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:52.214496 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:52.214498 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:52.215221 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:52.215222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:52.215765 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:52.215767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:52.216198 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:52.216199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:52.216264 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:52.216707 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:52.216711 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:52.218556 (+ 1845us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":142,"threads_started":1}
W20260809 08:52:52.218773 23805 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:52.213847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:55095 (local address 127.8.154.62:46079)
0809 08:52:52.213962 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:52.213965 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:52.214089 (+ 124us) server_negotiation.cc:407] Connection header received
0809 08:52:52.214167 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:52.214168 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:52.214183 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:52.214231 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:52.214590 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:52.215115 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:52.215855 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:52.216114 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:52.216197 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:52.218688 (+ 2491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.3:55095: BlockingRecv error: recv got EOF from 127.8.154.3:55095 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 09 08:52:52 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:52 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:52.942891 23808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:52.937675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.1:59719)
0809 08:52:52.937881 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:52.937892 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:52.937969 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:52.938135 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:52.938136 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:52.938317 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:52.938369 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:52.938371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:52.939054 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:52.939055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:52.939637 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:52.939639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:52.939988 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:52.939989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:52.940046 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:52.940641 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:52.940650 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:52.942817 (+ 2167us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":116,"threads_started":1}
W20260809 08:52:52.942997 23809 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:52.937812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:59719 (local address 127.8.154.62:46079)
0809 08:52:52.937920 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:52.937922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:52.937952 (+ 30us) server_negotiation.cc:407] Connection header received
0809 08:52:52.938017 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:52.938017 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:52.938028 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:52.938113 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:52.938441 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:52.938962 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:52.939715 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:52.939914 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:52.939968 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:52.942917 (+ 2949us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.1:59719: BlockingRecv error: recv got EOF from 127.8.154.1:59719 (error 108)
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":62,"threads_started":1}
Aug 09 08:52:53 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:53 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:53.074525 23810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:53.069534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.2:60575)
0809 08:52:53.069706 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:53.069718 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:53.069777 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:53.069961 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:53.069962 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:53.070240 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:53.070322 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:53.070325 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:53.071151 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:53.071152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:53.071689 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:53.071691 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:53.072024 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:53.072024 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:53.072113 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:53.072592 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:53.072597 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:53.074452 (+ 1855us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":105,"threads_started":1}
W20260809 08:52:53.074752 23809 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:53.069625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:60575 (local address 127.8.154.62:46079)
0809 08:52:53.069697 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:53.069699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:53.069779 (+ 80us) server_negotiation.cc:407] Connection header received
0809 08:52:53.069847 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:53.069848 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:53.069864 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:53.069912 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:53.070425 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:53.071075 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:53.071766 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:53.071953 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:53.072024 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:53.074565 (+ 2541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.2:60575: BlockingRecv error: recv got EOF from 127.8.154.2:60575 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 09 08:52:53 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.3@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:53 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.3@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:53.223939 23812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:53.219273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.3:48231)
0809 08:52:53.219440 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:53.219451 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:53.219526 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:53.219683 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:53.219684 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:53.219866 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:53.219915 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:53.219917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:53.220591 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:53.220592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:53.221137 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:53.221139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:53.221478 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:53.221479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:53.221526 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:53.222099 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:53.222104 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:53.223867 (+ 1763us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":105,"threads_started":1}
W20260809 08:52:53.224009 23809 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:53.219362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:48231 (local address 127.8.154.62:46079)
0809 08:52:53.219458 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:53.219460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:53.219519 (+ 59us) server_negotiation.cc:407] Connection header received
0809 08:52:53.219583 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:53.219584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:53.219595 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:53.219660 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:53.219990 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:53.220521 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:53.221218 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:53.221411 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:53.221466 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:53.223960 (+ 2494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.3:48231: BlockingRecv error: recv got EOF from 127.8.154.3:48231 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 09 08:52:53 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:53 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:53.948213 23813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:53.943399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.1:49805)
0809 08:52:53.943572 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:53.943584 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:53.943643 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:53.943786 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:53.943787 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:53.943981 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:53.944030 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:53.944032 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:53.944747 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:53.944748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:53.945278 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:53.945279 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:53.945648 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:53.945649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:53.945695 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:53.946253 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:53.946258 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:53.948087 (+ 1829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":97,"threads_started":1}
W20260809 08:52:53.948300 23814 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:53.943497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:49805 (local address 127.8.154.62:46079)
0809 08:52:53.943624 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:53.943631 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:53.943641 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:52:53.943703 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:53.943704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:53.943714 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:53.943765 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:53.944139 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:53.944674 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:53.945360 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:53.945573 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:53.945632 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:53.948226 (+ 2594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.1:49805: BlockingRecv error: recv got EOF from 127.8.154.1:49805 (error 108)
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":71,"threads_started":1}
Aug 09 08:52:54 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:54 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:54.079819 23815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:54.075116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.2:33253)
0809 08:52:54.075324 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:54.075341 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:54.075412 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:54.075552 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:54.075553 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:54.075729 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:54.075782 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:54.075784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:54.076492 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:54.076493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:54.077043 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:54.077045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:54.077375 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:54.077376 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:54.077421 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:54.077963 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:54.077968 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:54.079748 (+ 1780us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":108,"threads_started":1}
W20260809 08:52:54.079926 23814 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:54.075266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:33253 (local address 127.8.154.62:46079)
0809 08:52:54.075321 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:54.075323 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:54.075411 (+ 88us) server_negotiation.cc:407] Connection header received
0809 08:52:54.075459 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:54.075460 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:54.075470 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:54.075522 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:54.075879 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:54.076412 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:54.077118 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:54.077305 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:54.077358 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:54.079845 (+ 2487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.2:33253: BlockingRecv error: recv got EOF from 127.8.154.2:33253 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 09 08:52:54 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.3@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:54 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.3@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:54.229353 23817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:54.224475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.3:32979)
0809 08:52:54.224659 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:54.224671 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:54.224756 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:54.224921 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:54.224921 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:54.225122 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:54.225176 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:54.225178 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:54.225881 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:54.225883 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:54.226429 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:54.226431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:54.226803 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:54.226804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:54.226865 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:54.227427 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:54.227437 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:54.229282 (+ 1845us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":111,"threads_started":1}
W20260809 08:52:54.229421 23814 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:54.224558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:32979 (local address 127.8.154.62:46079)
0809 08:52:54.224639 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:54.224641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:54.224735 (+ 94us) server_negotiation.cc:407] Connection header received
0809 08:52:54.224816 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:54.224817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:54.224838 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:54.224910 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:54.225263 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:54.225784 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:54.226518 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:54.226723 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:54.226804 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:54.229374 (+ 2570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.3:32979: BlockingRecv error: recv got EOF from 127.8.154.3:32979 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 09 08:52:54 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:54 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:54.954080 23818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:54.948807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.1:43645)
0809 08:52:54.948958 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:54.948971 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:54.949062 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:54.949331 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:54.949332 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:54.949515 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:54.949569 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:54.949570 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:54.950242 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:54.950243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:54.951098 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:54.951101 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:54.951503 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:54.951505 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:54.951567 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:54.952118 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:54.952123 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:54.953967 (+ 1844us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":93,"threads_started":1}
W20260809 08:52:54.954133 23819 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:54.949029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:43645 (local address 127.8.154.62:46079)
0809 08:52:54.949167 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:54.949169 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:54.949177 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:52:54.949203 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:54.949203 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:54.949214 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:54.949288 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:54.949644 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:54.950165 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:54.951204 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:54.951420 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:54.951504 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:54.954056 (+ 2552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.1:43645: BlockingRecv error: recv got EOF from 127.8.154.1:43645 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":79,"threads_started":1}
Aug 09 08:52:55 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:55 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:55.085021 23820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:55.080365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.2:33521)
0809 08:52:55.080532 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:55.080543 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:55.080602 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:55.080742 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:55.080742 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:55.080922 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:55.080973 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:55.080974 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:55.081653 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:55.081654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:55.082200 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:55.082201 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:55.082524 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:55.082525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:55.082567 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:55.083038 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:55.083042 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:55.084959 (+ 1917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":100,"threads_started":1}
W20260809 08:52:55.085147 23819 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:55.080537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:33521 (local address 127.8.154.62:46079)
0809 08:52:55.080604 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:55.080606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:55.080638 (+ 32us) server_negotiation.cc:407] Connection header received
0809 08:52:55.080664 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:55.080665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:55.080674 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:55.080716 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:55.081057 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:55.081566 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:55.082273 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:55.082457 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:55.082509 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:55.085066 (+ 2557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.2:33521: BlockingRecv error: recv got EOF from 127.8.154.2:33521 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 09 08:52:55 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.3@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:55 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.3@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:55.235073 23821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:55.229935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.3:44927)
0809 08:52:55.230135 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:55.230147 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:55.230224 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:55.230437 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:55.230438 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:55.230670 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:55.230733 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:55.230736 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:55.231734 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:55.231735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:55.232328 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:55.232330 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:55.232667 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:55.232668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:55.232721 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:55.233208 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:55.233212 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:55.235000 (+ 1788us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":120,"threads_started":1}
W20260809 08:52:55.235239 23819 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:55.230032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:44927 (local address 127.8.154.62:46079)
0809 08:52:55.230097 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:55.230099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:55.230220 (+ 121us) server_negotiation.cc:407] Connection header received
0809 08:52:55.230317 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:55.230318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:55.230333 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:55.230425 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:55.230816 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:55.231659 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:55.232393 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:55.232581 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:55.232637 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:55.235151 (+ 2514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.3:44927: BlockingRecv error: recv got EOF from 127.8.154.3:44927 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 09 08:52:55 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:55 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:55.959242 23824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:55.954617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.1:44813)
0809 08:52:55.954755 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:55.954767 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:55.954829 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:55.954974 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:55.954976 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:55.955160 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:55.955211 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:55.955212 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:55.955855 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:55.955856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:55.956426 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:55.956428 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:55.956766 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:55.956767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:55.956815 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:55.957298 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:55.957302 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:55.959182 (+ 1880us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":80,"threads_started":1}
W20260809 08:52:55.959370 23827 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:55.954691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:44813 (local address 127.8.154.62:46079)
0809 08:52:55.954841 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:55.954843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:55.954851 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:52:55.954874 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:55.954875 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:55.954891 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:55.954957 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:55.955277 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:55.955782 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:55.956496 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:55.956696 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:55.956751 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:55.959282 (+ 2531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.1:44813: BlockingRecv error: recv got EOF from 127.8.154.1:44813 (error 108)
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":90,"threads_started":1}
Aug 09 08:52:56 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:56 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:56.094740 23828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:56.085538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.2:57389)
0809 08:52:56.085694 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:56.085705 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:56.085771 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:56.085998 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:56.086000 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:56.086190 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:56.086247 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:56.086249 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:56.086972 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:56.086973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:56.087535 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:56.087537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:56.087910 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:56.087911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:56.087958 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:56.092781 (+ 4823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:56.092786 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:56.094678 (+ 1892us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":95,"threads_started":1}
W20260809 08:52:56.094892 23827 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:56.085631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:57389 (local address 127.8.154.62:46079)
0809 08:52:56.085691 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:56.085693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:56.085760 (+ 67us) server_negotiation.cc:407] Connection header received
0809 08:52:56.085835 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:56.085836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:56.085846 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:56.085889 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:56.086356 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:56.086891 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:56.087622 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:56.087831 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:56.087892 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:56.094809 (+ 6917us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.2:57389: BlockingRecv error: recv got EOF from 127.8.154.2:57389 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 09 08:52:56 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.3@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:56 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.3@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:56.240564 23830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:56.235555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.3:38349)
0809 08:52:56.235740 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:56.235750 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:56.235812 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:56.235945 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:56.235945 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:56.236143 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:56.236192 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:56.236194 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:56.236837 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:56.236838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:56.237368 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:56.237369 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:56.237710 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:56.237710 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:56.237753 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:56.238240 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:56.238244 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:56.240399 (+ 2155us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":106,"threads_started":1}
W20260809 08:52:56.240553 23827 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:56.235641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:38349 (local address 127.8.154.62:46079)
0809 08:52:56.235691 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:56.235693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:56.235802 (+ 109us) server_negotiation.cc:407] Connection header received
0809 08:52:56.235872 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:56.235873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:56.235886 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:56.235929 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:56.236254 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:56.236767 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:56.237444 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:56.237644 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:56.237701 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:56.240494 (+ 2793us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.3:38349: BlockingRecv error: recv got EOF from 127.8.154.3:38349 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 09 08:52:56 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:56 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.1@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:56.964720 23835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:56.959828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.1:46839)
0809 08:52:56.960005 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:56.960016 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:56.960107 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:56.960272 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:56.960273 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:56.960444 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:56.960496 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:56.960497 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:56.961182 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:56.961183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:56.961718 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:56.961720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:56.962242 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:56.962243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:56.962318 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:56.962826 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:56.962831 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:56.964660 (+ 1829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":98,"threads_started":1}
W20260809 08:52:56.964874 23838 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:56.959932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:46839 (local address 127.8.154.62:46079)
0809 08:52:56.960136 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:56.960138 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:56.960149 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:52:56.960184 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:56.960185 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:56.960195 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:56.960258 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:56.960564 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:56.961105 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:56.961853 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:56.962157 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:56.962244 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:56.964783 (+ 2539us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.1:46839: BlockingRecv error: recv got EOF from 127.8.154.1:46839 (error 108)
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":73,"threads_started":1}
Aug 09 08:52:57 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:52:57 dist-test-slave-t3q3 krb5kdc[22275](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.8.154.2@KRBTEST.COM for oryx/127.8.154.62@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:52:57.099947 23839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:57.095263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46079 (local address 127.8.154.2:47513)
0809 08:52:57.095420 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:57.095431 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:57.095490 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:57.095656 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:57.095657 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:57.095830 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:57.095881 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:57.095882 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:57.096615 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:57.096616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:57.097165 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:57.097167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:57.097526 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:57.097527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:57.097568 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:57.098071 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:52:57.098076 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:57.099890 (+ 1814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.8.154.62:46079: Server oryx/127.8.154.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":94,"threads_started":1}
W20260809 08:52:57.100138 23838 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:57.095350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:47513 (local address 127.8.154.62:46079)
0809 08:52:57.095419 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:57.095421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:57.095476 (+ 55us) server_negotiation.cc:407] Connection header received
0809 08:52:57.095553 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:57.095554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:57.095564 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:57.095610 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:57.095960 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:57.096531 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:57.097252 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:57.097453 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:57.097517 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:57.100050 (+ 2533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.8.154.2:47513: BlockingRecv error: recv got EOF from 127.8.154.2:47513 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 08:52:57.155022 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 23146
I20260809 08:52:57.175817 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 23366
I20260809 08:52:57.183259 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 23561
I20260809 08:52:57.192265 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 23040
2026-08-09T08:52:57Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17766 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T08:52:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:52:57Z Disabled control of system clock
I20260809 08:52:57.230274 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:34753
--webserver_interface=127.8.154.62
--webserver_port=0
--builtin_ntp_servers=127.8.154.20:38223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:34753
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 08:52:57.312379 23846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:57.312584 23846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:57.312613 23846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:57.313737 23846 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:52:57.313773 23846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:57.313786 23846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:57.313798 23846 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:52:57.313810 23846 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:52:57.315078 23846 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:38223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:34753
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:34753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:57.315271 23846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:57.315496 23846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:57.320334 23858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:57.320369 23852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:57.320433 23846 server_base.cc:1034] running on GCE node
W20260809 08:52:57.320449 23855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:57.320806 23846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:57.321120 23846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:57.322280 23846 hybrid_clock.cc:648] HybridClock initialized: now 1786265577322273 us; error 39 us; skew 500 ppm
I20260809 08:52:57.323509 23846 webserver.cc:460] Webserver started at http://127.8.154.62:41861/ using document root <none> and password file <none>
I20260809 08:52:57.323745 23846 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:57.323813 23846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:57.323959 23846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:57.325033 23846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "d57007ff43bb4635bb37816371bb84cc"
format_stamp: "Formatted at 2026-08-09 08:52:57 on dist-test-slave-t3q3"
server_key: "b8384d59ef8a5d5e705ee0bf33c176b6"
server_key_iv: "5660eedd6dbc6710184bc86e3649d0e4"
server_key_version: "encryptionkey@0"
I20260809 08:52:57.325423 23846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "d57007ff43bb4635bb37816371bb84cc"
format_stamp: "Formatted at 2026-08-09 08:52:57 on dist-test-slave-t3q3"
server_key: "b8384d59ef8a5d5e705ee0bf33c176b6"
server_key_iv: "5660eedd6dbc6710184bc86e3649d0e4"
server_key_version: "encryptionkey@0"
I20260809 08:52:57.326785 23846 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 08:52:57.327540 23864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:57.327713 23846 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:52:57.327811 23846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "d57007ff43bb4635bb37816371bb84cc"
format_stamp: "Formatted at 2026-08-09 08:52:57 on dist-test-slave-t3q3"
server_key: "b8384d59ef8a5d5e705ee0bf33c176b6"
server_key_iv: "5660eedd6dbc6710184bc86e3649d0e4"
server_key_version: "encryptionkey@0"
I20260809 08:52:57.327890 23846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:57.341153 23846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:57.341511 23846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:57.341674 23846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:57.346458 23846 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:34753
I20260809 08:52:57.346486 23943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:34753 every 8 connection(s)
I20260809 08:52:57.346940 23846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:52:57.347733 23944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:57.350492 23944 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d57007ff43bb4635bb37816371bb84cc: Bootstrap starting.
I20260809 08:52:57.351133 23944 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d57007ff43bb4635bb37816371bb84cc: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:57.351441 23944 log.cc:824] T 00000000000000000000000000000000 P d57007ff43bb4635bb37816371bb84cc: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:57.352280 23944 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d57007ff43bb4635bb37816371bb84cc: No bootstrap required, opened a new log
I20260809 08:52:57.353876 23944 raft_consensus.cc:348] T 00000000000000000000000000000000 P d57007ff43bb4635bb37816371bb84cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d57007ff43bb4635bb37816371bb84cc" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 34753 } }
I20260809 08:52:57.354019 23944 raft_consensus.cc:374] T 00000000000000000000000000000000 P d57007ff43bb4635bb37816371bb84cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:57.354080 23944 raft_consensus.cc:729] T 00000000000000000000000000000000 P d57007ff43bb4635bb37816371bb84cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d57007ff43bb4635bb37816371bb84cc, State: Initialized, Role: FOLLOWER
I20260809 08:52:57.354228 23944 consensus_queue.cc:260] T 00000000000000000000000000000000 P d57007ff43bb4635bb37816371bb84cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d57007ff43bb4635bb37816371bb84cc" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 34753 } }
I20260809 08:52:57.354288 23944 raft_consensus.cc:388] T 00000000000000000000000000000000 P d57007ff43bb4635bb37816371bb84cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:52:57.354328 23944 raft_consensus.cc:482] T 00000000000000000000000000000000 P d57007ff43bb4635bb37816371bb84cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:52:57.354378 23944 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d57007ff43bb4635bb37816371bb84cc [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:57.354733 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 23846
I20260809 08:52:57.354838 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:52:57.355064 8808 external_mini_cluster.cc:1428] Setting key 92126773c5a077745a74ca9519eb5c9c
I20260809 08:52:57.355162 23944 raft_consensus.cc:504] T 00000000000000000000000000000000 P d57007ff43bb4635bb37816371bb84cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d57007ff43bb4635bb37816371bb84cc" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 34753 } }
I20260809 08:52:57.355275 23944 leader_election.cc:302] T 00000000000000000000000000000000 P d57007ff43bb4635bb37816371bb84cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d57007ff43bb4635bb37816371bb84cc; no voters:
I20260809 08:52:57.355450 23944 leader_election.cc:288] T 00000000000000000000000000000000 P d57007ff43bb4635bb37816371bb84cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:52:57.355510 23948 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d57007ff43bb4635bb37816371bb84cc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:52:57.355719 23944 sys_catalog.cc:564] T 00000000000000000000000000000000 P d57007ff43bb4635bb37816371bb84cc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:52:57.356033 23948 raft_consensus.cc:686] T 00000000000000000000000000000000 P d57007ff43bb4635bb37816371bb84cc [term 1 LEADER]: Becoming Leader. State: Replica: d57007ff43bb4635bb37816371bb84cc, State: Running, Role: LEADER
I20260809 08:52:57.356163 23948 consensus_queue.cc:237] T 00000000000000000000000000000000 P d57007ff43bb4635bb37816371bb84cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d57007ff43bb4635bb37816371bb84cc" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 34753 } }
I20260809 08:52:57.356529 23952 sys_catalog.cc:455] T 00000000000000000000000000000000 P d57007ff43bb4635bb37816371bb84cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d57007ff43bb4635bb37816371bb84cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d57007ff43bb4635bb37816371bb84cc" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 34753 } } }
I20260809 08:52:57.356627 23952 sys_catalog.cc:458] T 00000000000000000000000000000000 P d57007ff43bb4635bb37816371bb84cc [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:57.356982 23954 sys_catalog.cc:455] T 00000000000000000000000000000000 P d57007ff43bb4635bb37816371bb84cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader d57007ff43bb4635bb37816371bb84cc. Latest consensus state: current_term: 1 leader_uuid: "d57007ff43bb4635bb37816371bb84cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d57007ff43bb4635bb37816371bb84cc" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 34753 } } }
I20260809 08:52:57.357050 23954 sys_catalog.cc:458] T 00000000000000000000000000000000 P d57007ff43bb4635bb37816371bb84cc [sys.catalog]: This master's current role is: LEADER
W20260809 08:52:57.357846 23981 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d57007ff43bb4635bb37816371bb84cc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:52:57.357905 23981 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:52:57.357964 23971 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:52:57.358117 23971 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:52:57.360224 23971 catalog_manager.cc:1347] Generated new cluster ID: 3dd8cd200fa0402995b39b115066fc37
I20260809 08:52:57.360312 23971 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:52:57.361451 23956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:57.355912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39862 (local address 127.8.154.62:34753)
0809 08:52:57.356113 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:57.356123 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:57.356155 (+ 32us) server_negotiation.cc:407] Connection header received
0809 08:52:57.356289 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:57.356297 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:57.356370 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:57.356794 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:57.356905 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:57.358290 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:57.359274 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:57.359876 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:57.360013 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:57.360742 (+ 729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:57.360747 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:57.360747 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:52:57.360919 (+ 172us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:57.360955 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:52:57.360965 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:57.361043 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:57.361137 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 08:52:57.361302 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":64,"threads_started":1}
I20260809 08:52:57.372349 23971 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:52:57.372870 23971 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:52:57.379843 23971 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d57007ff43bb4635bb37816371bb84cc: Generated new TSK 0
I20260809 08:52:57.380033 23971 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:52:57.388351 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.1:0
--local_ip_for_outbound_sockets=127.8.154.1
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:34753
--builtin_ntp_servers=127.8.154.20:38223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 08:52:57.471518 23985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:57.471751 23985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:57.471788 23985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:57.473013 23985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:57.473074 23985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:57.473126 23985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.1
I20260809 08:52:57.474479 23985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:38223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:34753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.1
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:57.474740 23985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:57.475049 23985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:57.481097 24005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:57.481094 24000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:57.481098 24001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:57.481525 23985 server_base.cc:1034] running on GCE node
I20260809 08:52:57.481729 23985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:57.482007 23985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:57.483183 23985 hybrid_clock.cc:648] HybridClock initialized: now 1786265577483168 us; error 31 us; skew 500 ppm
I20260809 08:52:57.484594 23985 webserver.cc:460] Webserver started at http://127.8.154.1:39981/ using document root <none> and password file <none>
I20260809 08:52:57.484845 23985 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:57.484917 23985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:57.485080 23985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:57.486164 23985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/data/instance:
uuid: "c82acf52f8db41c2bd502905a7eee74a"
format_stamp: "Formatted at 2026-08-09 08:52:57 on dist-test-slave-t3q3"
server_key: "0000e4318af8fdc15a584fbcb7c06fcc"
server_key_iv: "08d8b6786bf80352d5704d00182f0dd6"
server_key_version: "encryptionkey@0"
I20260809 08:52:57.486544 23985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance:
uuid: "c82acf52f8db41c2bd502905a7eee74a"
format_stamp: "Formatted at 2026-08-09 08:52:57 on dist-test-slave-t3q3"
server_key: "0000e4318af8fdc15a584fbcb7c06fcc"
server_key_iv: "08d8b6786bf80352d5704d00182f0dd6"
server_key_version: "encryptionkey@0"
I20260809 08:52:57.487900 23985 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:52:57.488726 24016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:57.488937 23985 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:57.489034 23985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/wal
uuid: "c82acf52f8db41c2bd502905a7eee74a"
format_stamp: "Formatted at 2026-08-09 08:52:57 on dist-test-slave-t3q3"
server_key: "0000e4318af8fdc15a584fbcb7c06fcc"
server_key_iv: "08d8b6786bf80352d5704d00182f0dd6"
server_key_version: "encryptionkey@0"
I20260809 08:52:57.489116 23985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:57.510118 23985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:57.510473 23985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:57.510638 23985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:57.510902 23985 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:57.511240 23985 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:57.511292 23985 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:57.511333 23985 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:57.511363 23985 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:57.517518 23985 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.1:34029
I20260809 08:52:57.517601 24190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.1:34029 every 8 connection(s)
I20260809 08:52:57.517930 23985 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
I20260809 08:52:57.523056 23956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:57.518681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:59151 (local address 127.8.154.62:34753)
0809 08:52:57.518732 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:57.518734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:57.518980 (+ 246us) server_negotiation.cc:407] Connection header received
0809 08:52:57.519478 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:57.519479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:57.519491 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:57.519557 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:57.520258 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:57.520761 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:57.521729 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:57.521922 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:57.521980 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:57.522812 (+ 832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:57.522814 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:57.522814 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:52:57.522819 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:57.522833 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:52:57.522837 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:57.522885 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:57.522967 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 08:52:57.522995 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 08:52:57.523260 24195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:57.518639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:34753 (local address 127.8.154.1:59151)
0809 08:52:57.518837 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:57.518879 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:57.519389 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:57.519612 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:57.519615 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:57.519624 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:57.520180 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:57.520186 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:57.520841 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:57.520842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:57.521642 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:57.521644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:57.522021 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:57.522023 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:57.522143 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:57.522673 (+ 530us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:52:57.522673 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:57.522702 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:52:57.522724 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:57.522893 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:57.522898 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:57.522900 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 08:52:57.522952 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 08:52:57.523036 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":94,"threads_started":1}
I20260809 08:52:57.523820 24192 heartbeater.cc:347] Connected to a master server at 127.8.154.62:34753
I20260809 08:52:57.523921 24192 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:57.524121 24192 heartbeater.cc:511] Master 127.8.154.62:34753 requested a full tablet report, sending...
I20260809 08:52:57.524374 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 23985
I20260809 08:52:57.524482 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance
I20260809 08:52:57.524715 8808 external_mini_cluster.cc:1428] Setting key 2a2ace1ba0d2d7eb707265969dea45e6
I20260809 08:52:57.524734 23891 ts_manager.cc:194] Registered new tserver with Master: c82acf52f8db41c2bd502905a7eee74a (127.8.154.1:34029)
I20260809 08:52:57.525435 23891 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.154.1:59151
I20260809 08:52:57.525959 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.2:0
--local_ip_for_outbound_sockets=127.8.154.2
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:34753
--builtin_ntp_servers=127.8.154.20:38223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 08:52:57.613270 24196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:57.613543 24196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:57.613583 24196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:57.615123 24196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:57.615183 24196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:57.615226 24196 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.2
I20260809 08:52:57.616672 24196 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:38223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:34753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.2
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:57.616930 24196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:57.617233 24196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:57.622359 24206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:57.622426 24209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:57.622393 24204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:57.622614 24196 server_base.cc:1034] running on GCE node
I20260809 08:52:57.622830 24196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:57.623085 24196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:57.624248 24196 hybrid_clock.cc:648] HybridClock initialized: now 1786265577624239 us; error 37 us; skew 500 ppm
I20260809 08:52:57.625607 24196 webserver.cc:460] Webserver started at http://127.8.154.2:33123/ using document root <none> and password file <none>
I20260809 08:52:57.625849 24196 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:57.625918 24196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:57.626077 24196 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:57.627233 24196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/data/instance:
uuid: "b942b1871e8d4d9dbe54843a6206f708"
format_stamp: "Formatted at 2026-08-09 08:52:57 on dist-test-slave-t3q3"
server_key: "f2366688a629a0cee14ebbb1a12aa33d"
server_key_iv: "d4c563d5892962fc3c097885485b7e92"
server_key_version: "encryptionkey@0"
I20260809 08:52:57.627619 24196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance:
uuid: "b942b1871e8d4d9dbe54843a6206f708"
format_stamp: "Formatted at 2026-08-09 08:52:57 on dist-test-slave-t3q3"
server_key: "f2366688a629a0cee14ebbb1a12aa33d"
server_key_iv: "d4c563d5892962fc3c097885485b7e92"
server_key_version: "encryptionkey@0"
I20260809 08:52:57.629082 24196 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:52:57.629802 24217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:57.629987 24196 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:52:57.630093 24196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/wal
uuid: "b942b1871e8d4d9dbe54843a6206f708"
format_stamp: "Formatted at 2026-08-09 08:52:57 on dist-test-slave-t3q3"
server_key: "f2366688a629a0cee14ebbb1a12aa33d"
server_key_iv: "d4c563d5892962fc3c097885485b7e92"
server_key_version: "encryptionkey@0"
I20260809 08:52:57.630167 24196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:57.646430 24196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:57.646759 24196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:57.646917 24196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:57.647166 24196 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:57.647523 24196 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:57.647575 24196 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:57.647620 24196 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:57.647646 24196 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:57.653756 24196 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.2:42879
I20260809 08:52:57.653847 24391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.2:42879 every 8 connection(s)
I20260809 08:52:57.654162 24196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
I20260809 08:52:57.659801 23956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:57.654968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:36699 (local address 127.8.154.62:34753)
0809 08:52:57.655082 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:57.655085 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:57.655285 (+ 200us) server_negotiation.cc:407] Connection header received
0809 08:52:57.655853 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:57.655855 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:57.655868 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:57.655933 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:57.656798 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:57.657331 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:57.658406 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:57.658626 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:57.658702 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:57.659490 (+ 788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:57.659491 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:57.659491 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:52:57.659496 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:57.659513 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:52:57.659517 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:57.659594 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:57.659691 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 08:52:57.659722 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 08:52:57.660027 24396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:57.654882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:34753 (local address 127.8.154.2:36699)
0809 08:52:57.655161 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:57.655195 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:57.655739 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:57.656045 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:57.656050 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:57.656080 (+ 30us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:57.656687 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:57.656713 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:57.657426 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:57.657427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:57.658306 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:57.658309 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:57.658710 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:57.658712 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:57.658847 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:57.659344 (+ 497us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:52:57.659344 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:57.659374 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:52:57.659396 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:57.659599 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:57.659605 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:57.659608 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:52:57.659691 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 08:52:57.659795 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":89,"threads_started":1}
I20260809 08:52:57.660518 24394 heartbeater.cc:347] Connected to a master server at 127.8.154.62:34753
I20260809 08:52:57.660626 24394 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:57.660808 24394 heartbeater.cc:511] Master 127.8.154.62:34753 requested a full tablet report, sending...
I20260809 08:52:57.661185 23891 ts_manager.cc:194] Registered new tserver with Master: b942b1871e8d4d9dbe54843a6206f708 (127.8.154.2:42879)
I20260809 08:52:57.661556 23891 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.154.2:36699
I20260809 08:52:57.661931 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 24196
I20260809 08:52:57.662043 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance
I20260809 08:52:57.662235 8808 external_mini_cluster.cc:1428] Setting key d81c4ca28c038ae4cb64919b8b008917
I20260809 08:52:57.663045 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.3:0
--local_ip_for_outbound_sockets=127.8.154.3
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:34753
--builtin_ntp_servers=127.8.154.20:38223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 08:52:57.746008 24397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:57.746212 24397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:57.746233 24397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:57.747331 24397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:57.747372 24397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:57.747397 24397 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.3
I20260809 08:52:57.748711 24397 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:38223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:34753
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24397
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.3
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:57.748939 24397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:57.749239 24397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:57.754268 24405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:57.754257 24404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:57.754403 24397 server_base.cc:1034] running on GCE node
W20260809 08:52:57.754257 24408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:57.754743 24397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:57.754989 24397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:57.756143 24397 hybrid_clock.cc:648] HybridClock initialized: now 1786265577756121 us; error 58 us; skew 500 ppm
I20260809 08:52:57.757498 24397 webserver.cc:460] Webserver started at http://127.8.154.3:45123/ using document root <none> and password file <none>
I20260809 08:52:57.757731 24397 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:57.757797 24397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:57.757933 24397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:57.758955 24397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/data/instance:
uuid: "3025eb42f8744695bc7516fa2150b490"
format_stamp: "Formatted at 2026-08-09 08:52:57 on dist-test-slave-t3q3"
server_key: "345c7c9c53f1b77fa361d08670eacbf7"
server_key_iv: "e1bc73d29df812048d7fc8c647741d24"
server_key_version: "encryptionkey@0"
I20260809 08:52:57.759318 24397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance:
uuid: "3025eb42f8744695bc7516fa2150b490"
format_stamp: "Formatted at 2026-08-09 08:52:57 on dist-test-slave-t3q3"
server_key: "345c7c9c53f1b77fa361d08670eacbf7"
server_key_iv: "e1bc73d29df812048d7fc8c647741d24"
server_key_version: "encryptionkey@0"
I20260809 08:52:57.760789 24397 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:52:57.761505 24417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:57.761674 24397 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:52:57.761771 24397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/wal
uuid: "3025eb42f8744695bc7516fa2150b490"
format_stamp: "Formatted at 2026-08-09 08:52:57 on dist-test-slave-t3q3"
server_key: "345c7c9c53f1b77fa361d08670eacbf7"
server_key_iv: "e1bc73d29df812048d7fc8c647741d24"
server_key_version: "encryptionkey@0"
I20260809 08:52:57.761847 24397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:57.783538 24397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:57.783872 24397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:57.784030 24397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:57.784308 24397 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:57.784647 24397 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:57.784699 24397 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:57.784736 24397 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:57.784780 24397 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:57.790723 24397 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.3:39611
I20260809 08:52:57.790769 24601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.3:39611 every 8 connection(s)
I20260809 08:52:57.791139 24397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
I20260809 08:52:57.796248 23956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:57.791869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:36069 (local address 127.8.154.62:34753)
0809 08:52:57.791938 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:57.791940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:57.792212 (+ 272us) server_negotiation.cc:407] Connection header received
0809 08:52:57.792720 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:57.792721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:57.792731 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:57.792791 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:57.793421 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:57.793911 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:57.794894 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:57.795080 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:57.795134 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:57.795965 (+ 831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:57.795966 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:57.795966 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:52:57.795970 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:57.795984 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:52:57.795988 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:57.796033 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:57.796164 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 08:52:57.796190 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 08:52:57.796430 24606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:57.791871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:34753 (local address 127.8.154.3:36069)
0809 08:52:57.792094 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:57.792135 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:57.792625 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:57.792846 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:57.792849 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:57.792858 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:57.793349 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:57.793354 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:57.794004 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:57.794005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:57.794804 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:52:57.794806 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:52:57.795154 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:52:57.795155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:52:57.795284 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:57.795834 (+ 550us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:52:57.795835 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:57.795865 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:52:57.795888 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:57.796044 (+ 156us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:57.796049 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:57.796051 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 08:52:57.796155 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 08:52:57.796236 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":87,"threads_started":1}
I20260809 08:52:57.796885 24602 heartbeater.cc:347] Connected to a master server at 127.8.154.62:34753
I20260809 08:52:57.796980 24602 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:57.797166 24602 heartbeater.cc:511] Master 127.8.154.62:34753 requested a full tablet report, sending...
I20260809 08:52:57.797511 23891 ts_manager.cc:194] Registered new tserver with Master: 3025eb42f8744695bc7516fa2150b490 (127.8.154.3:39611)
I20260809 08:52:57.797861 23891 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.154.3:36069
I20260809 08:52:57.798681 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 24397
I20260809 08:52:57.798759 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance
I20260809 08:52:57.798910 8808 external_mini_cluster.cc:1428] Setting key 1e7656b679db9d55894bfaac5ac0e1dd
I20260809 08:52:57.799721 8808 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 08:52:57.801589 23956 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:57.801017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39866 (local address 127.8.154.62:34753)
0809 08:52:57.801105 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:57.801107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:57.801209 (+ 102us) server_negotiation.cc:407] Connection header received
0809 08:52:57.801284 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:57.801285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:57.801298 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:57.801364 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:57.801527 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39866: BlockingRecv error: recv got EOF from 127.0.0.1:39866 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 08:52:57.801674 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 23985
I20260809 08:52:57.817526 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 24196
I20260809 08:52:57.831898 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 24397
I20260809 08:52:57.844798 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 23846
2026-08-09T08:52:57Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (646 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T08:52:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:52:57Z Disabled control of system clock
I20260809 08:52:57.876852 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:37437
--webserver_interface=127.8.154.62
--webserver_port=0
--builtin_ntp_servers=127.8.154.20:40171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:37437
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 08:52:57.959359 24622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:57.959560 24622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:57.959582 24622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:57.960668 24622 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:52:57.960708 24622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:57.960721 24622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:57.960733 24622 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:52:57.960745 24622 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:52:57.961997 24622 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:40171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:37437
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:37437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:57.962198 24622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:57.962435 24622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:57.967327 24629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:57.967348 24634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:57.967500 24628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:57.967469 24622 server_base.cc:1034] running on GCE node
I20260809 08:52:57.967952 24622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:57.968274 24622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:57.969446 24622 hybrid_clock.cc:648] HybridClock initialized: now 1786265577969428 us; error 38 us; skew 500 ppm
I20260809 08:52:57.970808 24622 webserver.cc:460] Webserver started at http://127.8.154.62:43243/ using document root <none> and password file <none>
I20260809 08:52:57.971047 24622 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:57.971096 24622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:57.971266 24622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:57.972378 24622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "2028de0d4a87408285bbc0b7050c190f"
format_stamp: "Formatted at 2026-08-09 08:52:57 on dist-test-slave-t3q3"
server_key: "0d1c3a1f7615af4adb7dfe2b7d990d0b"
server_key_iv: "b31da81ba37759c802575b0f321e0d44"
server_key_version: "encryptionkey@0"
I20260809 08:52:57.972745 24622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "2028de0d4a87408285bbc0b7050c190f"
format_stamp: "Formatted at 2026-08-09 08:52:57 on dist-test-slave-t3q3"
server_key: "0d1c3a1f7615af4adb7dfe2b7d990d0b"
server_key_iv: "b31da81ba37759c802575b0f321e0d44"
server_key_version: "encryptionkey@0"
I20260809 08:52:57.974071 24622 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:52:57.974895 24647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:57.975085 24622 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:57.975193 24622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "2028de0d4a87408285bbc0b7050c190f"
format_stamp: "Formatted at 2026-08-09 08:52:57 on dist-test-slave-t3q3"
server_key: "0d1c3a1f7615af4adb7dfe2b7d990d0b"
server_key_iv: "b31da81ba37759c802575b0f321e0d44"
server_key_version: "encryptionkey@0"
I20260809 08:52:57.975265 24622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:57.978716 24622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:57.978988 24622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:57.979131 24622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:57.983064 24622 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:37437
I20260809 08:52:57.983115 24737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:37437 every 8 connection(s)
I20260809 08:52:57.983529 24622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:52:57.984352 24739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:57.987154 24739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2028de0d4a87408285bbc0b7050c190f: Bootstrap starting.
I20260809 08:52:57.987696 24739 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2028de0d4a87408285bbc0b7050c190f: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:57.987938 24739 log.cc:824] T 00000000000000000000000000000000 P 2028de0d4a87408285bbc0b7050c190f: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:57.988668 24739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2028de0d4a87408285bbc0b7050c190f: No bootstrap required, opened a new log
I20260809 08:52:57.990185 24739 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2028de0d4a87408285bbc0b7050c190f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2028de0d4a87408285bbc0b7050c190f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 37437 } }
I20260809 08:52:57.990324 24739 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2028de0d4a87408285bbc0b7050c190f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:57.990347 24739 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2028de0d4a87408285bbc0b7050c190f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2028de0d4a87408285bbc0b7050c190f, State: Initialized, Role: FOLLOWER
I20260809 08:52:57.990481 24739 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2028de0d4a87408285bbc0b7050c190f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2028de0d4a87408285bbc0b7050c190f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 37437 } }
I20260809 08:52:57.990569 24739 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2028de0d4a87408285bbc0b7050c190f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:52:57.990593 24739 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2028de0d4a87408285bbc0b7050c190f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:52:57.990621 24739 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2028de0d4a87408285bbc0b7050c190f [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:57.991165 24739 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2028de0d4a87408285bbc0b7050c190f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2028de0d4a87408285bbc0b7050c190f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 37437 } }
I20260809 08:52:57.991255 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 24622
I20260809 08:52:57.991284 24739 leader_election.cc:302] T 00000000000000000000000000000000 P 2028de0d4a87408285bbc0b7050c190f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2028de0d4a87408285bbc0b7050c190f; no voters:
I20260809 08:52:57.991357 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:52:57.991451 24739 leader_election.cc:288] T 00000000000000000000000000000000 P 2028de0d4a87408285bbc0b7050c190f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:52:57.991600 8808 external_mini_cluster.cc:1428] Setting key 273610355c3f8560f157d40157b32721
I20260809 08:52:57.991658 24746 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2028de0d4a87408285bbc0b7050c190f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:52:57.991734 24739 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2028de0d4a87408285bbc0b7050c190f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:52:57.992231 24746 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2028de0d4a87408285bbc0b7050c190f [term 1 LEADER]: Becoming Leader. State: Replica: 2028de0d4a87408285bbc0b7050c190f, State: Running, Role: LEADER
I20260809 08:52:57.992413 24746 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2028de0d4a87408285bbc0b7050c190f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2028de0d4a87408285bbc0b7050c190f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 37437 } }
I20260809 08:52:57.993044 24745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2028de0d4a87408285bbc0b7050c190f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2028de0d4a87408285bbc0b7050c190f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2028de0d4a87408285bbc0b7050c190f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 37437 } } }
I20260809 08:52:57.993160 24745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2028de0d4a87408285bbc0b7050c190f [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:57.993690 24750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2028de0d4a87408285bbc0b7050c190f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2028de0d4a87408285bbc0b7050c190f. Latest consensus state: current_term: 1 leader_uuid: "2028de0d4a87408285bbc0b7050c190f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2028de0d4a87408285bbc0b7050c190f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 37437 } } }
I20260809 08:52:57.993870 24750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2028de0d4a87408285bbc0b7050c190f [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:57.993980 24769 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:52:57.994448 24753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:57.992176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38894 (local address 127.8.154.62:37437)
0809 08:52:57.992756 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:57.992767 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:57.992802 (+ 35us) server_negotiation.cc:407] Connection header received
0809 08:52:57.992932 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:57.992940 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:57.993009 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:57.993451 (+ 442us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:57.993944 (+ 493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:57.993948 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:57.993948 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:52:57.993952 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:57.993996 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:52:57.994010 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:57.994134 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:57.994215 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 08:52:57.994274 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":77,"threads_started":1}
I20260809 08:52:57.994613 24769 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:52:57.996181 24769 catalog_manager.cc:1347] Generated new cluster ID: d8157e4998a043a18da0db287c500056
I20260809 08:52:57.996256 24769 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:52:58.007119 24769 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:52:58.007313 24769 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:52:58.014484 24769 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2028de0d4a87408285bbc0b7050c190f: Generated new TSK 0
I20260809 08:52:58.014714 24769 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:52:58.019735 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.1:0
--local_ip_for_outbound_sockets=127.8.154.1
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:37437
--builtin_ntp_servers=127.8.154.20:40171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 08:52:58.118722 24781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:58.118916 24781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:58.118938 24781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:58.120101 24781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:58.120141 24781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:58.120165 24781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.1
I20260809 08:52:58.121479 24781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:40171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:37437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.1
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:58.121697 24781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:58.121989 24781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:58.127224 24787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:58.127286 24781 server_base.cc:1034] running on GCE node
W20260809 08:52:58.127344 24791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:58.127221 24793 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:58.127671 24781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:58.127930 24781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:58.129097 24781 hybrid_clock.cc:648] HybridClock initialized: now 1786265578129110 us; error 55 us; skew 500 ppm
I20260809 08:52:58.130483 24781 webserver.cc:460] Webserver started at http://127.8.154.1:36623/ using document root <none> and password file <none>
I20260809 08:52:58.130784 24781 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:58.130862 24781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:58.131029 24781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:58.132467 24781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/data/instance:
uuid: "992610253eeb41f0a8fa342eadd8c770"
format_stamp: "Formatted at 2026-08-09 08:52:58 on dist-test-slave-t3q3"
server_key: "a34e6a9b03b1c0e3f30fb82f78c4b474"
server_key_iv: "94232acf30787196c148e1b85cf1fa7f"
server_key_version: "encryptionkey@0"
I20260809 08:52:58.132897 24781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance:
uuid: "992610253eeb41f0a8fa342eadd8c770"
format_stamp: "Formatted at 2026-08-09 08:52:58 on dist-test-slave-t3q3"
server_key: "a34e6a9b03b1c0e3f30fb82f78c4b474"
server_key_iv: "94232acf30787196c148e1b85cf1fa7f"
server_key_version: "encryptionkey@0"
I20260809 08:52:58.134654 24781 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260809 08:52:58.135530 24805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:58.135733 24781 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:58.135815 24781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/wal
uuid: "992610253eeb41f0a8fa342eadd8c770"
format_stamp: "Formatted at 2026-08-09 08:52:58 on dist-test-slave-t3q3"
server_key: "a34e6a9b03b1c0e3f30fb82f78c4b474"
server_key_iv: "94232acf30787196c148e1b85cf1fa7f"
server_key_version: "encryptionkey@0"
I20260809 08:52:58.135895 24781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:58.139103 24781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:58.139367 24781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:58.139513 24781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:58.139739 24781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:58.140046 24781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:58.140121 24781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:58.140161 24781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:58.140187 24781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:58.145857 24781 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.1:45419
I20260809 08:52:58.145892 24981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.1:45419 every 8 connection(s)
I20260809 08:52:58.146234 24781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
I20260809 08:52:58.149142 24753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:58.146966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:44601 (local address 127.8.154.62:37437)
0809 08:52:58.147040 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:58.147042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:58.147274 (+ 232us) server_negotiation.cc:407] Connection header received
0809 08:52:58.147741 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:58.147742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:58.147748 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:58.147787 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:58.148965 (+ 1178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:58.148966 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:58.148966 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:52:58.148969 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:58.148980 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:52:58.148983 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:58.149021 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:58.149078 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 08:52:58.149090 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 08:52:58.149349 24987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:58.146943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37437 (local address 127.8.154.1:44601)
0809 08:52:58.147156 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:58.147190 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:58.147644 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:58.147844 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:58.147847 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:58.147855 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:58.148833 (+ 978us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:52:58.148833 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:58.148861 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:52:58.148889 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:58.149021 (+ 132us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:58.149023 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:58.149026 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:52:58.149077 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 08:52:58.149126 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":116,"threads_started":1}
I20260809 08:52:58.149905 24982 heartbeater.cc:347] Connected to a master server at 127.8.154.62:37437
I20260809 08:52:58.149996 24982 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:58.150216 24982 heartbeater.cc:511] Master 127.8.154.62:37437 requested a full tablet report, sending...
I20260809 08:52:58.150735 24673 ts_manager.cc:194] Registered new tserver with Master: 992610253eeb41f0a8fa342eadd8c770 (127.8.154.1:45419)
I20260809 08:52:58.156128 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 24781
I20260809 08:52:58.156239 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance
I20260809 08:52:58.156451 8808 external_mini_cluster.cc:1428] Setting key 896440b1299beac9d925920552ee9e5e
I20260809 08:52:58.157423 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.2:0
--local_ip_for_outbound_sockets=127.8.154.2
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:37437
--builtin_ntp_servers=127.8.154.20:40171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 08:52:58.244095 24988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:58.244359 24988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:58.244396 24988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:58.245544 24988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:58.245606 24988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:58.245651 24988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.2
I20260809 08:52:58.247012 24988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:40171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:37437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.2
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:58.247278 24988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:58.247577 24988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:58.252545 24995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:58.252591 24998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:58.252660 24988 server_base.cc:1034] running on GCE node
W20260809 08:52:58.252718 24993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:58.252945 24988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:58.253208 24988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:58.254361 24988 hybrid_clock.cc:648] HybridClock initialized: now 1786265578254349 us; error 28 us; skew 500 ppm
I20260809 08:52:58.255692 24988 webserver.cc:460] Webserver started at http://127.8.154.2:44587/ using document root <none> and password file <none>
I20260809 08:52:58.255955 24988 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:58.256003 24988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:58.256168 24988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:58.257234 24988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/data/instance:
uuid: "8332ecfe2cc34642b44257e060fc159c"
format_stamp: "Formatted at 2026-08-09 08:52:58 on dist-test-slave-t3q3"
server_key: "d62b79bfaa8aa0e5fec1fe23e4a7d110"
server_key_iv: "6c4e39639af730072e5b2e6c1fc65f4c"
server_key_version: "encryptionkey@0"
I20260809 08:52:58.257612 24988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance:
uuid: "8332ecfe2cc34642b44257e060fc159c"
format_stamp: "Formatted at 2026-08-09 08:52:58 on dist-test-slave-t3q3"
server_key: "d62b79bfaa8aa0e5fec1fe23e4a7d110"
server_key_iv: "6c4e39639af730072e5b2e6c1fc65f4c"
server_key_version: "encryptionkey@0"
I20260809 08:52:58.259196 24988 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:52:58.259970 25008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:58.260161 24988 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:52:58.260267 24988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/wal
uuid: "8332ecfe2cc34642b44257e060fc159c"
format_stamp: "Formatted at 2026-08-09 08:52:58 on dist-test-slave-t3q3"
server_key: "d62b79bfaa8aa0e5fec1fe23e4a7d110"
server_key_iv: "6c4e39639af730072e5b2e6c1fc65f4c"
server_key_version: "encryptionkey@0"
I20260809 08:52:58.260339 24988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:58.263721 24988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:58.263994 24988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:58.264159 24988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:58.264366 24988 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:58.264685 24988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:58.264737 24988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:58.264777 24988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:58.264804 24988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:58.270937 24988 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.2:35935
I20260809 08:52:58.271013 25175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.2:35935 every 8 connection(s)
I20260809 08:52:58.271338 24988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
I20260809 08:52:58.273033 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 24988
I20260809 08:52:58.273182 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance
I20260809 08:52:58.273382 8808 external_mini_cluster.cc:1428] Setting key fc01539580a08acfd4ebd409ce8dfb3a
I20260809 08:52:58.274427 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.3:0
--local_ip_for_outbound_sockets=127.8.154.3
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:37437
--builtin_ntp_servers=127.8.154.20:40171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 08:52:58.274827 24753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:58.272194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:58833 (local address 127.8.154.62:37437)
0809 08:52:58.272258 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:58.272260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:58.272500 (+ 240us) server_negotiation.cc:407] Connection header received
0809 08:52:58.273072 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:58.273073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:58.273080 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:58.273134 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:58.274559 (+ 1425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:58.274560 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:58.274560 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:52:58.274565 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:58.274583 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:52:58.274588 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:58.274656 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:58.274738 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 08:52:58.274754 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 08:52:58.275029 25183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:58.272169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37437 (local address 127.8.154.2:58833)
0809 08:52:58.272380 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:58.272416 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:58.272929 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:58.273228 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:58.273232 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:58.273256 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:58.274390 (+ 1134us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:52:58.274390 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:58.274421 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:52:58.274450 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:58.274654 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:58.274661 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:58.274665 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:52:58.274748 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 08:52:58.274813 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":103,"threads_started":1}
I20260809 08:52:58.275511 25177 heartbeater.cc:347] Connected to a master server at 127.8.154.62:37437
I20260809 08:52:58.275614 25177 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:58.275764 25177 heartbeater.cc:511] Master 127.8.154.62:37437 requested a full tablet report, sending...
I20260809 08:52:58.276173 24673 ts_manager.cc:194] Registered new tserver with Master: 8332ecfe2cc34642b44257e060fc159c (127.8.154.2:35935)
W20260809 08:52:58.360304 25184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:58.360504 25184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:58.360526 25184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:58.361640 25184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:58.361675 25184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:58.361701 25184 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.3
I20260809 08:52:58.363015 25184 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:40171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:37437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25184
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.3
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:58.363209 25184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:58.363454 25184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:52:58.368537 25195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:58.368537 25197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:58.368669 25184 server_base.cc:1034] running on GCE node
W20260809 08:52:58.368578 25194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:58.368968 25184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:58.369226 25184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:58.370390 25184 hybrid_clock.cc:648] HybridClock initialized: now 1786265578370378 us; error 41 us; skew 500 ppm
I20260809 08:52:58.371688 25184 webserver.cc:460] Webserver started at http://127.8.154.3:43509/ using document root <none> and password file <none>
I20260809 08:52:58.371929 25184 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:58.371999 25184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:58.372174 25184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:58.373274 25184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/data/instance:
uuid: "3ba2449a367347d991c1b83a4437c85f"
format_stamp: "Formatted at 2026-08-09 08:52:58 on dist-test-slave-t3q3"
server_key: "a150f5ce870d1ec47d276f50d9745325"
server_key_iv: "9ac8ce0bc6e04e69b7d14640be86c21a"
server_key_version: "encryptionkey@0"
I20260809 08:52:58.373648 25184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance:
uuid: "3ba2449a367347d991c1b83a4437c85f"
format_stamp: "Formatted at 2026-08-09 08:52:58 on dist-test-slave-t3q3"
server_key: "a150f5ce870d1ec47d276f50d9745325"
server_key_iv: "9ac8ce0bc6e04e69b7d14640be86c21a"
server_key_version: "encryptionkey@0"
I20260809 08:52:58.375116 25184 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:52:58.375851 25206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:58.376106 25184 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:58.376217 25184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/wal
uuid: "3ba2449a367347d991c1b83a4437c85f"
format_stamp: "Formatted at 2026-08-09 08:52:58 on dist-test-slave-t3q3"
server_key: "a150f5ce870d1ec47d276f50d9745325"
server_key_iv: "9ac8ce0bc6e04e69b7d14640be86c21a"
server_key_version: "encryptionkey@0"
I20260809 08:52:58.376296 25184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:58.379616 25184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:58.379879 25184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:58.380013 25184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:58.380259 25184 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:52:58.380569 25184 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:52:58.380625 25184 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:58.380664 25184 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:52:58.380692 25184 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:58.386345 25184 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.3:41411
I20260809 08:52:58.386381 25385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.3:41411 every 8 connection(s)
I20260809 08:52:58.386727 25184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
I20260809 08:52:58.389758 24753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:58.387527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:43327 (local address 127.8.154.62:37437)
0809 08:52:58.387584 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:58.387586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:58.387791 (+ 205us) server_negotiation.cc:407] Connection header received
0809 08:52:58.388323 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:58.388325 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:58.388330 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:58.388401 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:58.389570 (+ 1169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:58.389571 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:58.389571 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:52:58.389575 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:58.389589 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:52:58.389593 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:58.389639 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:58.389693 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 08:52:58.389705 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 08:52:58.389981 25389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:58.387482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37437 (local address 127.8.154.3:43327)
0809 08:52:58.387670 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 08:52:58.387710 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 08:52:58.388224 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:52:58.388457 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:52:58.388460 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:52:58.388468 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:52:58.389424 (+ 956us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:52:58.389424 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:52:58.389452 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:52:58.389481 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:52:58.389645 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:52:58.389647 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:52:58.389649 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 08:52:58.389698 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 08:52:58.389744 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":92,"threads_started":1}
I20260809 08:52:58.390532 25386 heartbeater.cc:347] Connected to a master server at 127.8.154.62:37437
I20260809 08:52:58.390560 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 25184
I20260809 08:52:58.390651 25386 heartbeater.cc:464] Registering TS with master...
I20260809 08:52:58.390662 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance
I20260809 08:52:58.390799 25386 heartbeater.cc:511] Master 127.8.154.62:37437 requested a full tablet report, sending...
I20260809 08:52:58.390846 8808 external_mini_cluster.cc:1428] Setting key 8b7adfe4ad2734ee570d457af35e790f
I20260809 08:52:58.391180 24673 ts_manager.cc:194] Registered new tserver with Master: 3ba2449a367347d991c1b83a4437c85f (127.8.154.3:41411)
I20260809 08:52:58.391781 8808 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 08:52:58.393920 24753 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:52:58.393430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38900 (local address 127.8.154.62:37437)
0809 08:52:58.393472 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:58.393474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:58.393622 (+ 148us) server_negotiation.cc:407] Connection header received
0809 08:52:58.393664 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:58.393664 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:58.393669 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:58.393715 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:58.393838 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38900: BlockingRecv error: recv got EOF from 127.0.0.1:38900 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 08:52:58.393949 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 24781
I20260809 08:52:58.409096 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 24988
I20260809 08:52:58.419571 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 25184
I20260809 08:52:58.429387 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 24622
2026-08-09T08:52:58Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (584 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:52:58 dist-test-slave-t3q3 krb5kdc[25399](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:52:58 dist-test-slave-t3q3 krb5kdc[25399](info): set up 2 sockets
Aug 09 08:52:58 dist-test-slave-t3q3 krb5kdc[25399](info): commencing operation
krb5kdc: starting...
W20260809 08:52:59.679323 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.227s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:52:59 dist-test-slave-t3q3 krb5kdc[25399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:52:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:52:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
I20260809 08:52:59.802351 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:41743
--webserver_interface=127.8.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:35939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:41743
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:52:59.885111 25418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:52:59.885322 25418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:52:59.885344 25418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:52:59.886416 25418 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:52:59.886451 25418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:52:59.886466 25418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:52:59.886477 25418 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:52:59.886489 25418 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:52:59.887902 25418 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:35939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:41743
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:41743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:52:59.888190 25418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:52:59.888388 25418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 08:52:59.893400 25418 server_base.cc:1034] running on GCE node
W20260809 08:52:59.893556 25423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:59.893364 25432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:52:59.893378 25425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:52:59.893883 25418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:52:59.894192 25418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:52:59.895336 25418 hybrid_clock.cc:648] HybridClock initialized: now 1786265579895320 us; error 38 us; skew 500 ppm
Aug 09 08:52:59 dist-test-slave-t3q3 krb5kdc[25399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265579, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:52:59.897930 25418 init.cc:377] Logged in from keytab as kudu/127.8.154.62@KRBTEST.COM (short username kudu)
I20260809 08:52:59.898398 25418 webserver.cc:460] Webserver started at http://127.8.154.62:34797/ using document root <none> and password file <none>
I20260809 08:52:59.898625 25418 fs_manager.cc:359] Metadata directory not provided
I20260809 08:52:59.898691 25418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:52:59.898838 25418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:52:59.899899 25418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "1a66cc87d1284a3fac21d5c90b7328a0"
format_stamp: "Formatted at 2026-08-09 08:52:59 on dist-test-slave-t3q3"
server_key: "08e5e9394490f9f3e09a66a9a83f8a25"
server_key_iv: "83a3006d5bb0b63a76701ced1343971f"
server_key_version: "encryptionkey@0"
I20260809 08:52:59.900358 25418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "1a66cc87d1284a3fac21d5c90b7328a0"
format_stamp: "Formatted at 2026-08-09 08:52:59 on dist-test-slave-t3q3"
server_key: "08e5e9394490f9f3e09a66a9a83f8a25"
server_key_iv: "83a3006d5bb0b63a76701ced1343971f"
server_key_version: "encryptionkey@0"
I20260809 08:52:59.901808 25418 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:59.902576 25440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:52:59.902770 25418 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:52:59.902871 25418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "1a66cc87d1284a3fac21d5c90b7328a0"
format_stamp: "Formatted at 2026-08-09 08:52:59 on dist-test-slave-t3q3"
server_key: "08e5e9394490f9f3e09a66a9a83f8a25"
server_key_iv: "83a3006d5bb0b63a76701ced1343971f"
server_key_version: "encryptionkey@0"
I20260809 08:52:59.902947 25418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:52:59.920717 25418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:52:59.923025 25418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:52:59.923234 25418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:52:59.927413 25418 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:41743
I20260809 08:52:59.927451 25522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:41743 every 8 connection(s)
I20260809 08:52:59.927815 25418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:52:59.928867 25524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:52:59.931173 25524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a66cc87d1284a3fac21d5c90b7328a0: Bootstrap starting.
I20260809 08:52:59.931866 25524 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a66cc87d1284a3fac21d5c90b7328a0: Neither blocks nor log segments found. Creating new log.
I20260809 08:52:59.932238 25524 log.cc:824] T 00000000000000000000000000000000 P 1a66cc87d1284a3fac21d5c90b7328a0: Log is configured to *not* fsync() on all Append() calls
I20260809 08:52:59.934285 25524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a66cc87d1284a3fac21d5c90b7328a0: No bootstrap required, opened a new log
I20260809 08:52:59.935881 25524 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a66cc87d1284a3fac21d5c90b7328a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a66cc87d1284a3fac21d5c90b7328a0" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 41743 } }
I20260809 08:52:59.936021 25524 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a66cc87d1284a3fac21d5c90b7328a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:52:59.936044 25524 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a66cc87d1284a3fac21d5c90b7328a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a66cc87d1284a3fac21d5c90b7328a0, State: Initialized, Role: FOLLOWER
I20260809 08:52:59.936233 25524 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a66cc87d1284a3fac21d5c90b7328a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a66cc87d1284a3fac21d5c90b7328a0" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 41743 } }
I20260809 08:52:59.936312 25524 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a66cc87d1284a3fac21d5c90b7328a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:52:59.936364 25524 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a66cc87d1284a3fac21d5c90b7328a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:52:59.936414 25524 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a66cc87d1284a3fac21d5c90b7328a0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:52:59.937223 25524 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a66cc87d1284a3fac21d5c90b7328a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a66cc87d1284a3fac21d5c90b7328a0" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 41743 } }
I20260809 08:52:59.937340 25524 leader_election.cc:302] T 00000000000000000000000000000000 P 1a66cc87d1284a3fac21d5c90b7328a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a66cc87d1284a3fac21d5c90b7328a0; no voters:
I20260809 08:52:59.937502 25524 leader_election.cc:288] T 00000000000000000000000000000000 P 1a66cc87d1284a3fac21d5c90b7328a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:52:59.937522 25529 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a66cc87d1284a3fac21d5c90b7328a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:52:59.937537 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 25418
I20260809 08:52:59.937654 25529 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a66cc87d1284a3fac21d5c90b7328a0 [term 1 LEADER]: Becoming Leader. State: Replica: 1a66cc87d1284a3fac21d5c90b7328a0, State: Running, Role: LEADER
I20260809 08:52:59.937666 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:52:59.937760 25524 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a66cc87d1284a3fac21d5c90b7328a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:52:59.937740 25529 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a66cc87d1284a3fac21d5c90b7328a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a66cc87d1284a3fac21d5c90b7328a0" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 41743 } }
I20260809 08:52:59.937887 8808 external_mini_cluster.cc:1428] Setting key 22cfc3136ebad3d9cab04c838215a00f
I20260809 08:52:59.938098 25531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a66cc87d1284a3fac21d5c90b7328a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a66cc87d1284a3fac21d5c90b7328a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a66cc87d1284a3fac21d5c90b7328a0" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 41743 } } }
I20260809 08:52:59.938198 25531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a66cc87d1284a3fac21d5c90b7328a0 [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:59.938092 25529 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a66cc87d1284a3fac21d5c90b7328a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a66cc87d1284a3fac21d5c90b7328a0. Latest consensus state: current_term: 1 leader_uuid: "1a66cc87d1284a3fac21d5c90b7328a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a66cc87d1284a3fac21d5c90b7328a0" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 41743 } } }
I20260809 08:52:59.938398 25529 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a66cc87d1284a3fac21d5c90b7328a0 [sys.catalog]: This master's current role is: LEADER
I20260809 08:52:59.939150 25534 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:52:59.939795 25534 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:52:59.942183 25534 catalog_manager.cc:1347] Generated new cluster ID: cc1435bb0ffb433aaa953a39ec8fa4e1
I20260809 08:52:59.942241 25534 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 08:52:59 dist-test-slave-t3q3 krb5kdc[25399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:52:59.948933 25544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:52:59.938579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60178 (local address 127.8.154.62:41743)
0809 08:52:59.938784 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 08:52:59.938794 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:52:59.938892 (+ 98us) server_negotiation.cc:407] Connection header received
0809 08:52:59.939104 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:52:59.939113 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:52:59.939176 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:52:59.939323 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:52:59.940253 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:59.941640 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:59.942439 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:52:59.942870 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:52:59.942984 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:52:59.945303 (+ 2319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:52:59.945307 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:52:59.945307 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:52:59.945319 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:52:59.947372 (+ 2053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:59.947741 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:59.947744 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:59.947745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:59.947846 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:52:59.948084 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:52:59.948085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:52:59.948085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:52:59.948320 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:52:59.948494 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 08:52:59.948584 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 08:52:59.948726 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":103,"threads_started":1}
I20260809 08:52:59.956359 25534 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:52:59.957041 25534 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:52:59.963893 25534 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a66cc87d1284a3fac21d5c90b7328a0: Generated new TSK 0
I20260809 08:52:59.964079 25534 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab.
Entry for principal kudu/127.8.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab.
I20260809 08:52:59.991451 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.1:0
--local_ip_for_outbound_sockets=127.8.154.1
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:41743
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:35939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:53:00.077100 25561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:53:00.077297 25561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:53:00.077322 25561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:53:00.078450 25561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:53:00.078491 25561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:53:00.078539 25561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.1
I20260809 08:53:00.079969 25561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:35939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:41743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.1
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:53:00.080245 25561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:53:00.080473 25561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:53:00.085635 25569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:53:00.085724 25561 server_base.cc:1034] running on GCE node
W20260809 08:53:00.085661 25572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:53:00.085887 25578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:53:00.086107 25561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:53:00.086315 25561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:53:00.087455 25561 hybrid_clock.cc:648] HybridClock initialized: now 1786265580087444 us; error 30 us; skew 500 ppm
Aug 09 08:53:00 dist-test-slave-t3q3 krb5kdc[25399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265580, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:53:00.090027 25561 init.cc:377] Logged in from keytab as kudu/127.8.154.1@KRBTEST.COM (short username kudu)
I20260809 08:53:00.090408 25561 webserver.cc:460] Webserver started at http://127.8.154.1:42449/ using document root <none> and password file <none>
I20260809 08:53:00.090603 25561 fs_manager.cc:359] Metadata directory not provided
I20260809 08:53:00.090642 25561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:53:00.090764 25561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:53:00.091760 25561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/data/instance:
uuid: "d565d39be5cb46d0b3a42bb91f89e557"
format_stamp: "Formatted at 2026-08-09 08:53:00 on dist-test-slave-t3q3"
server_key: "5375b9b0bef96b1aeb98468331bc4900"
server_key_iv: "66ddaab8a8568249489e7761210a5299"
server_key_version: "encryptionkey@0"
I20260809 08:53:00.092124 25561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance:
uuid: "d565d39be5cb46d0b3a42bb91f89e557"
format_stamp: "Formatted at 2026-08-09 08:53:00 on dist-test-slave-t3q3"
server_key: "5375b9b0bef96b1aeb98468331bc4900"
server_key_iv: "66ddaab8a8568249489e7761210a5299"
server_key_version: "encryptionkey@0"
I20260809 08:53:00.093420 25561 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:53:00.094162 25587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:53:00.094429 25561 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:53:00.094507 25561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/wal
uuid: "d565d39be5cb46d0b3a42bb91f89e557"
format_stamp: "Formatted at 2026-08-09 08:53:00 on dist-test-slave-t3q3"
server_key: "5375b9b0bef96b1aeb98468331bc4900"
server_key_iv: "66ddaab8a8568249489e7761210a5299"
server_key_version: "encryptionkey@0"
I20260809 08:53:00.094573 25561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:53:00.119803 25561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:53:00.122006 25561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:53:00.122160 25561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:53:00.122434 25561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:53:00.122789 25561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:53:00.122845 25561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:53:00.122888 25561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:53:00.122916 25561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:53:00.129048 25561 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.1:38389
I20260809 08:53:00.129094 25753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.1:38389 every 8 connection(s)
I20260809 08:53:00.129459 25561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
Aug 09 08:53:00 dist-test-slave-t3q3 krb5kdc[25399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265580, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:53:00.137615 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 25561
I20260809 08:53:00.137722 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance
I20260809 08:53:00.137979 8808 external_mini_cluster.cc:1428] Setting key 795f939a94d34130c1b26ca91b96632a
I20260809 08:53:00.139016 25544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:00.130293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:58927 (local address 127.8.154.62:41743)
0809 08:53:00.130368 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 08:53:00.130370 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:53:00.130570 (+ 200us) server_negotiation.cc:407] Connection header received
0809 08:53:00.131061 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:53:00.131062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:53:00.131072 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:53:00.131151 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:53:00.132123 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:00.132632 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:00.133600 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:00.133789 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:00.133841 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:53:00.135942 (+ 2101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:53:00.135944 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:53:00.135944 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:53:00.135950 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:53:00.137288 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:53:00.137748 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:53:00.137750 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:53:00.137750 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:53:00.137850 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:53:00.138158 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:53:00.138159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:53:00.138160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:53:00.138337 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:53:00.138428 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 08:53:00.138782 (+ 354us) server_negotiation.cc:299] Negotiation successful
0809 08:53:00.138943 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 08:53:00.139286 25758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:00.130251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:41743 (local address 127.8.154.1:58927)
0809 08:53:00.130439 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 08:53:00.130481 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 08:53:00.130979 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:53:00.131233 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:53:00.131236 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:53:00.131813 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:53:00.132011 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:53:00.132015 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:53:00.132711 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:53:00.132712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:53:00.133525 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:53:00.133526 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:53:00.133859 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:53:00.133860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:53:00.133959 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:53:00.134489 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:53:00.134497 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:53:00.135858 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:53:00.137357 (+ 1499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:53:00.137363 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:53:00.137365 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:53:00.137641 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:53:00.137923 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:53:00.137924 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:53:00.137924 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:53:00.138057 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:53:00.138420 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:53:00.138428 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:53:00.138588 (+ 160us) client_negotiation.cc:769] Sending connection context
0809 08:53:00.138744 (+ 156us) client_negotiation.cc:240] Negotiation successful
0809 08:53:00.138932 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":109,"threads_started":1}
I20260809 08:53:00.139868 25754 heartbeater.cc:347] Connected to a master server at 127.8.154.62:41743
I20260809 08:53:00.139976 25754 heartbeater.cc:464] Registering TS with master...
I20260809 08:53:00.140187 25754 heartbeater.cc:511] Master 127.8.154.62:41743 requested a full tablet report, sending...
I20260809 08:53:00.140805 25470 ts_manager.cc:194] Registered new tserver with Master: d565d39be5cb46d0b3a42bb91f89e557 (127.8.154.1:38389)
I20260809 08:53:00.141402 25470 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.1@KRBTEST.COM'} at 127.8.154.1:58927
WARNING: no policy specified for kudu/127.8.154.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab.
Entry for principal kudu/127.8.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab.
I20260809 08:53:00.164825 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.2:0
--local_ip_for_outbound_sockets=127.8.154.2
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:41743
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:35939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:53:00.249209 25761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:53:00.249444 25761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:53:00.249487 25761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:53:00.250615 25761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:53:00.250674 25761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:53:00.250717 25761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.2
I20260809 08:53:00.252221 25761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:35939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:41743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.2
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:53:00.252492 25761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:53:00.252717 25761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:53:00.257731 25770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:53:00.257763 25767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:53:00.257793 25761 server_base.cc:1034] running on GCE node
W20260809 08:53:00.257733 25766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:53:00.258196 25761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:53:00.258479 25761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:53:00.259667 25761 hybrid_clock.cc:648] HybridClock initialized: now 1786265580259655 us; error 24 us; skew 500 ppm
Aug 09 08:53:00 dist-test-slave-t3q3 krb5kdc[25399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265580, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:53:00.262372 25761 init.cc:377] Logged in from keytab as kudu/127.8.154.2@KRBTEST.COM (short username kudu)
I20260809 08:53:00.262787 25761 webserver.cc:460] Webserver started at http://127.8.154.2:41395/ using document root <none> and password file <none>
I20260809 08:53:00.263017 25761 fs_manager.cc:359] Metadata directory not provided
I20260809 08:53:00.263082 25761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:53:00.263236 25761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:53:00.264326 25761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/data/instance:
uuid: "3b84b24f6e6547adb247c40d40fa439c"
format_stamp: "Formatted at 2026-08-09 08:53:00 on dist-test-slave-t3q3"
server_key: "4e010c5ae4bafd6cbbce110a591da509"
server_key_iv: "e5a5f36c14a0aedcce62c9e8ec284a2e"
server_key_version: "encryptionkey@0"
I20260809 08:53:00.264686 25761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance:
uuid: "3b84b24f6e6547adb247c40d40fa439c"
format_stamp: "Formatted at 2026-08-09 08:53:00 on dist-test-slave-t3q3"
server_key: "4e010c5ae4bafd6cbbce110a591da509"
server_key_iv: "e5a5f36c14a0aedcce62c9e8ec284a2e"
server_key_version: "encryptionkey@0"
I20260809 08:53:00.266142 25761 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:53:00.266862 25781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:53:00.267089 25761 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:53:00.267204 25761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/wal
uuid: "3b84b24f6e6547adb247c40d40fa439c"
format_stamp: "Formatted at 2026-08-09 08:53:00 on dist-test-slave-t3q3"
server_key: "4e010c5ae4bafd6cbbce110a591da509"
server_key_iv: "e5a5f36c14a0aedcce62c9e8ec284a2e"
server_key_version: "encryptionkey@0"
I20260809 08:53:00.267274 25761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:53:00.276579 25761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:53:00.278836 25761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:53:00.278999 25761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:53:00.279227 25761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:53:00.279534 25761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:53:00.279584 25761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:53:00.279618 25761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:53:00.279644 25761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:53:00.285571 25761 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.2:32865
I20260809 08:53:00.285640 25955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.2:32865 every 8 connection(s)
I20260809 08:53:00.286031 25761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
I20260809 08:53:00.290295 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 25761
I20260809 08:53:00.290412 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance
I20260809 08:53:00.290618 8808 external_mini_cluster.cc:1428] Setting key 642b2670ce90d74691e43b2073378f23
Aug 09 08:53:00 dist-test-slave-t3q3 krb5kdc[25399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265580, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:53:00.295706 25544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:00.286802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:57553 (local address 127.8.154.62:41743)
0809 08:53:00.286892 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0809 08:53:00.286895 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:53:00.287189 (+ 294us) server_negotiation.cc:407] Connection header received
0809 08:53:00.287553 (+ 364us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:53:00.287554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:53:00.287565 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:53:00.287621 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:53:00.288621 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:00.289144 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:00.290165 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:00.290362 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:00.290441 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:53:00.292999 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:53:00.293000 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:53:00.293001 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:53:00.293007 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:53:00.294339 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:53:00.294739 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:53:00.294740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:53:00.294740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:53:00.294801 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:53:00.295052 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:53:00.295053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:53:00.295054 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:53:00.295172 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:53:00.295241 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 08:53:00.295521 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 08:53:00.295615 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 08:53:00.295900 25958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:00.286753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:41743 (local address 127.8.154.2:57553)
0809 08:53:00.286938 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 08:53:00.286975 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 08:53:00.287468 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:53:00.287683 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:53:00.287686 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:53:00.288335 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:53:00.288533 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:53:00.288539 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:53:00.289239 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:53:00.289240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:53:00.290037 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:53:00.290039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:53:00.290487 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:53:00.290488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:53:00.290608 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:53:00.291238 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:53:00.291248 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:53:00.292879 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:53:00.294422 (+ 1543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:53:00.294429 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:53:00.294432 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:53:00.294664 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:53:00.294869 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:53:00.294870 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:53:00.294870 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:53:00.294980 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:53:00.295244 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:53:00.295251 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:53:00.295387 (+ 136us) client_negotiation.cc:769] Sending connection context
0809 08:53:00.295487 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 08:53:00.295657 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":108,"threads_started":1}
I20260809 08:53:00.296391 25956 heartbeater.cc:347] Connected to a master server at 127.8.154.62:41743
I20260809 08:53:00.296480 25956 heartbeater.cc:464] Registering TS with master...
I20260809 08:53:00.296669 25956 heartbeater.cc:511] Master 127.8.154.62:41743 requested a full tablet report, sending...
I20260809 08:53:00.297025 25470 ts_manager.cc:194] Registered new tserver with Master: 3b84b24f6e6547adb247c40d40fa439c (127.8.154.2:32865)
I20260809 08:53:00.297465 25470 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.2@KRBTEST.COM'} at 127.8.154.2:57553
WARNING: no policy specified for kudu/127.8.154.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab.
Entry for principal kudu/127.8.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab.
I20260809 08:53:00.316708 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.3:0
--local_ip_for_outbound_sockets=127.8.154.3
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:41743
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:35939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:53:00.401151 25961 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:53:00.401356 25961 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:53:00.401379 25961 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:53:00.402611 25961 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:53:00.402657 25961 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:53:00.402683 25961 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.3
I20260809 08:53:00.404201 25961 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:35939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:41743
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25961
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.3
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:53:00.404448 25961 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:53:00.404675 25961 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:53:00.409965 25966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:53:00.409970 25969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:53:00.410099 25967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:53:00.410382 25961 server_base.cc:1034] running on GCE node
I20260809 08:53:00.410593 25961 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:53:00.410835 25961 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:53:00.412006 25961 hybrid_clock.cc:648] HybridClock initialized: now 1786265580411986 us; error 36 us; skew 500 ppm
Aug 09 08:53:00 dist-test-slave-t3q3 krb5kdc[25399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265580, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:53:00.414687 25961 init.cc:377] Logged in from keytab as kudu/127.8.154.3@KRBTEST.COM (short username kudu)
I20260809 08:53:00.415086 25961 webserver.cc:460] Webserver started at http://127.8.154.3:33915/ using document root <none> and password file <none>
I20260809 08:53:00.415290 25961 fs_manager.cc:359] Metadata directory not provided
I20260809 08:53:00.415330 25961 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:53:00.415426 25961 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:53:00.416507 25961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/data/instance:
uuid: "561728ee3b7b4ad5ba647c9f56d2bef2"
format_stamp: "Formatted at 2026-08-09 08:53:00 on dist-test-slave-t3q3"
server_key: "993be8b5822179bd2d0a1a79ff114260"
server_key_iv: "861ce0e2680299f532d3848a08122225"
server_key_version: "encryptionkey@0"
I20260809 08:53:00.416915 25961 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance:
uuid: "561728ee3b7b4ad5ba647c9f56d2bef2"
format_stamp: "Formatted at 2026-08-09 08:53:00 on dist-test-slave-t3q3"
server_key: "993be8b5822179bd2d0a1a79ff114260"
server_key_iv: "861ce0e2680299f532d3848a08122225"
server_key_version: "encryptionkey@0"
I20260809 08:53:00.418184 25961 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:53:00.418860 25980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:53:00.419044 25961 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:53:00.419129 25961 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/wal
uuid: "561728ee3b7b4ad5ba647c9f56d2bef2"
format_stamp: "Formatted at 2026-08-09 08:53:00 on dist-test-slave-t3q3"
server_key: "993be8b5822179bd2d0a1a79ff114260"
server_key_iv: "861ce0e2680299f532d3848a08122225"
server_key_version: "encryptionkey@0"
I20260809 08:53:00.419184 25961 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:53:00.434090 25961 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:53:00.435961 25961 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:53:00.436133 25961 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:53:00.436327 25961 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:53:00.436590 25961 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:53:00.436630 25961 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:53:00.436650 25961 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:53:00.436663 25961 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:53:00.442688 25961 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.3:32845
I20260809 08:53:00.442761 26157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.3:32845 every 8 connection(s)
I20260809 08:53:00.443077 25961 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
Aug 09 08:53:00 dist-test-slave-t3q3 krb5kdc[25399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265580, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:53:00.451705 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 25961
I20260809 08:53:00.451812 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance
I20260809 08:53:00.452003 8808 external_mini_cluster.cc:1428] Setting key b311c29fa80b539707203053d53b684a
I20260809 08:53:00.452821 25544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:00.443804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:54385 (local address 127.8.154.62:41743)
0809 08:53:00.443873 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 08:53:00.443876 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:53:00.444115 (+ 239us) server_negotiation.cc:407] Connection header received
0809 08:53:00.444646 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:53:00.444647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:53:00.444657 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:53:00.444705 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:53:00.445642 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:00.446137 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:00.447162 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:00.447344 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:00.447395 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:53:00.449749 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:53:00.449751 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:53:00.449751 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:53:00.449759 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:53:00.451171 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:53:00.451598 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:53:00.451599 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:53:00.451600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:53:00.451669 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:53:00.451989 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:53:00.451990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:53:00.451991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:53:00.452202 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:53:00.452285 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 08:53:00.452648 (+ 363us) server_negotiation.cc:299] Negotiation successful
0809 08:53:00.452731 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 08:53:00.453033 26162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:00.443800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:41743 (local address 127.8.154.3:54385)
0809 08:53:00.443978 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 08:53:00.444015 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 08:53:00.444548 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:53:00.444770 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:53:00.444773 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:53:00.445356 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:53:00.445558 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:53:00.445563 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:53:00.446216 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:53:00.446217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:53:00.447045 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:53:00.447047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:53:00.447414 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:53:00.447415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:53:00.447528 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:53:00.448114 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:53:00.448126 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:53:00.449642 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:53:00.451246 (+ 1604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:53:00.451252 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:53:00.451253 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:53:00.451502 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:53:00.451755 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:53:00.451756 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:53:00.451756 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:53:00.451915 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:53:00.452347 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:53:00.452354 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:53:00.452496 (+ 142us) client_negotiation.cc:769] Sending connection context
0809 08:53:00.452611 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 08:53:00.452774 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":108,"threads_started":1}
I20260809 08:53:00.453500 26158 heartbeater.cc:347] Connected to a master server at 127.8.154.62:41743
I20260809 08:53:00.453605 26158 heartbeater.cc:464] Registering TS with master...
I20260809 08:53:00.453784 26158 heartbeater.cc:511] Master 127.8.154.62:41743 requested a full tablet report, sending...
I20260809 08:53:00.454187 25470 ts_manager.cc:194] Registered new tserver with Master: 561728ee3b7b4ad5ba647c9f56d2bef2 (127.8.154.3:32845)
I20260809 08:53:00.454545 25470 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.3@KRBTEST.COM'} at 127.8.154.3:54385
I20260809 08:53:00.463917 8808 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:53:00.471873 25544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:00.465446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60190 (local address 127.8.154.62:41743)
0809 08:53:00.465546 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0809 08:53:00.465548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:53:00.465613 (+ 65us) server_negotiation.cc:407] Connection header received
0809 08:53:00.465705 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:53:00.465705 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:53:00.465716 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:53:00.465774 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:53:00.466384 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:00.466886 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:00.467644 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:00.467845 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:00.467921 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:53:00.469206 (+ 1285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:53:00.469208 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:53:00.469208 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:53:00.469216 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:53:00.470658 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:53:00.471007 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:53:00.471008 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:53:00.471009 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:53:00.471070 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:53:00.471299 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:53:00.471300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:53:00.471301 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:53:00.471460 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:53:00.471566 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 08:53:00.471675 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 08:53:00.471760 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 08:53:00.473805 25470 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60190:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:53:00.475404 25470 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:53:00.481791 26179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:00.477785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54844 (local address 127.8.154.1:38389)
0809 08:53:00.477997 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 08:53:00.478002 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:53:00.478124 (+ 122us) server_negotiation.cc:407] Connection header received
0809 08:53:00.478203 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:53:00.478207 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:53:00.478224 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:53:00.478310 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:53:00.478529 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:00.479306 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:00.480599 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:00.481345 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:00.481382 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:53:00.481423 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 08:53:00.481663 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 08:53:00.481703 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":82,"threads_started":1}
I20260809 08:53:00.481777 26180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:00.477661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:38389 (local address 127.0.0.1:54844)
0809 08:53:00.478050 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0809 08:53:00.478061 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:53:00.478130 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:53:00.478321 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:53:00.478325 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:53:00.478327 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:53:00.478404 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:53:00.478406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:53:00.479414 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:53:00.479415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:53:00.480468 (+ 1053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:53:00.480471 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:53:00.481473 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:53:00.481473 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:53:00.481553 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:53:00.481563 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 08:53:00.481643 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 08:53:00.481702 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":101,"threads_started":1}
I20260809 08:53:00.481899 26175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:00.477872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:32845 (local address 127.0.0.1:51320)
0809 08:53:00.478078 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 08:53:00.478088 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:53:00.478137 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:53:00.478424 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:53:00.478425 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:53:00.478428 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:53:00.478487 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:53:00.478489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:53:00.479821 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:53:00.479822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:53:00.480826 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:53:00.480828 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:53:00.481625 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:53:00.481627 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:53:00.481689 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:53:00.481694 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:53:00.481726 (+ 32us) client_negotiation.cc:240] Negotiation successful
0809 08:53:00.481766 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":1,"thread_start_us":68,"threads_started":1}
I20260809 08:53:00.482198 26187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:00.478089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51320 (local address 127.8.154.3:32845)
0809 08:53:00.478268 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 08:53:00.478273 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:53:00.478283 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:53:00.478322 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:53:00.478324 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:53:00.478340 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:53:00.478425 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:53:00.478577 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:00.479702 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:00.480909 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:00.481513 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:00.481981 (+ 468us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:53:00.482055 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 08:53:00.482082 (+ 27us) server_negotiation.cc:299] Negotiation successful
0809 08:53:00.482121 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":107,"threads_started":1}
I20260809 08:53:00.482774 26178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:00.477644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:32865 (local address 127.0.0.1:38518)
0809 08:53:00.477822 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 08:53:00.477865 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 08:53:00.477939 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:53:00.478715 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:53:00.478716 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:53:00.478718 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:53:00.478751 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:53:00.478752 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:53:00.480018 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:53:00.480019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:53:00.481092 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:53:00.481095 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:53:00.482533 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:53:00.482534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:53:00.482599 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:53:00.482603 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:53:00.482671 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 08:53:00.482711 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":62,"threads_started":1}
I20260809 08:53:00.482637 25650 tablet_service.cc:1467] Processing CreateTablet for tablet 2883c249cea34a9bab50fa2a6ddbce96 (DEFAULT_TABLE table=test-table [id=c40f04d70a21477a851a83045a991225]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:53:00.482921 26055 tablet_service.cc:1467] Processing CreateTablet for tablet 2883c249cea34a9bab50fa2a6ddbce96 (DEFAULT_TABLE table=test-table [id=c40f04d70a21477a851a83045a991225]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:53:00.483065 26184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:00.477842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38518 (local address 127.8.154.2:32865)
0809 08:53:00.478272 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0809 08:53:00.478277 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:53:00.478287 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:53:00.478327 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:53:00.478329 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:53:00.478342 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:53:00.478649 (+ 307us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:53:00.478877 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:00.479916 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:00.481186 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:00.482434 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:00.482531 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:53:00.482586 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 08:53:00.482928 (+ 342us) server_negotiation.cc:299] Negotiation successful
0809 08:53:00.482980 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":105,"threads_started":1}
I20260809 08:53:00.483073 25650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2883c249cea34a9bab50fa2a6ddbce96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:53:00.483282 26055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2883c249cea34a9bab50fa2a6ddbce96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:53:00.483685 25860 tablet_service.cc:1467] Processing CreateTablet for tablet 2883c249cea34a9bab50fa2a6ddbce96 (DEFAULT_TABLE table=test-table [id=c40f04d70a21477a851a83045a991225]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:53:00.484023 25860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2883c249cea34a9bab50fa2a6ddbce96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:53:00.486014 26191 tablet_bootstrap.cc:492] T 2883c249cea34a9bab50fa2a6ddbce96 P 561728ee3b7b4ad5ba647c9f56d2bef2: Bootstrap starting.
I20260809 08:53:00.486689 26191 tablet_bootstrap.cc:654] T 2883c249cea34a9bab50fa2a6ddbce96 P 561728ee3b7b4ad5ba647c9f56d2bef2: Neither blocks nor log segments found. Creating new log.
I20260809 08:53:00.486680 26192 tablet_bootstrap.cc:492] T 2883c249cea34a9bab50fa2a6ddbce96 P d565d39be5cb46d0b3a42bb91f89e557: Bootstrap starting.
I20260809 08:53:00.487314 26193 tablet_bootstrap.cc:492] T 2883c249cea34a9bab50fa2a6ddbce96 P 3b84b24f6e6547adb247c40d40fa439c: Bootstrap starting.
I20260809 08:53:00.487355 26191 log.cc:824] T 2883c249cea34a9bab50fa2a6ddbce96 P 561728ee3b7b4ad5ba647c9f56d2bef2: Log is configured to *not* fsync() on all Append() calls
I20260809 08:53:00.487614 26192 tablet_bootstrap.cc:654] T 2883c249cea34a9bab50fa2a6ddbce96 P d565d39be5cb46d0b3a42bb91f89e557: Neither blocks nor log segments found. Creating new log.
I20260809 08:53:00.487939 26193 tablet_bootstrap.cc:654] T 2883c249cea34a9bab50fa2a6ddbce96 P 3b84b24f6e6547adb247c40d40fa439c: Neither blocks nor log segments found. Creating new log.
I20260809 08:53:00.488224 26192 log.cc:824] T 2883c249cea34a9bab50fa2a6ddbce96 P d565d39be5cb46d0b3a42bb91f89e557: Log is configured to *not* fsync() on all Append() calls
I20260809 08:53:00.488560 26193 log.cc:824] T 2883c249cea34a9bab50fa2a6ddbce96 P 3b84b24f6e6547adb247c40d40fa439c: Log is configured to *not* fsync() on all Append() calls
I20260809 08:53:00.488845 26191 tablet_bootstrap.cc:492] T 2883c249cea34a9bab50fa2a6ddbce96 P 561728ee3b7b4ad5ba647c9f56d2bef2: No bootstrap required, opened a new log
I20260809 08:53:00.488945 26191 ts_tablet_manager.cc:1397] T 2883c249cea34a9bab50fa2a6ddbce96 P 561728ee3b7b4ad5ba647c9f56d2bef2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 08:53:00.489056 26192 tablet_bootstrap.cc:492] T 2883c249cea34a9bab50fa2a6ddbce96 P d565d39be5cb46d0b3a42bb91f89e557: No bootstrap required, opened a new log
I20260809 08:53:00.489122 26192 ts_tablet_manager.cc:1397] T 2883c249cea34a9bab50fa2a6ddbce96 P d565d39be5cb46d0b3a42bb91f89e557: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 08:53:00.489357 26193 tablet_bootstrap.cc:492] T 2883c249cea34a9bab50fa2a6ddbce96 P 3b84b24f6e6547adb247c40d40fa439c: No bootstrap required, opened a new log
I20260809 08:53:00.489441 26193 ts_tablet_manager.cc:1397] T 2883c249cea34a9bab50fa2a6ddbce96 P 3b84b24f6e6547adb247c40d40fa439c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 08:53:00.490926 26193 raft_consensus.cc:348] T 2883c249cea34a9bab50fa2a6ddbce96 P 3b84b24f6e6547adb247c40d40fa439c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b84b24f6e6547adb247c40d40fa439c" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 32865 } } peers { permanent_uuid: "d565d39be5cb46d0b3a42bb91f89e557" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 38389 } } peers { permanent_uuid: "561728ee3b7b4ad5ba647c9f56d2bef2" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 32845 } }
I20260809 08:53:00.490972 26191 raft_consensus.cc:348] T 2883c249cea34a9bab50fa2a6ddbce96 P 561728ee3b7b4ad5ba647c9f56d2bef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b84b24f6e6547adb247c40d40fa439c" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 32865 } } peers { permanent_uuid: "d565d39be5cb46d0b3a42bb91f89e557" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 38389 } } peers { permanent_uuid: "561728ee3b7b4ad5ba647c9f56d2bef2" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 32845 } }
I20260809 08:53:00.491098 26193 raft_consensus.cc:374] T 2883c249cea34a9bab50fa2a6ddbce96 P 3b84b24f6e6547adb247c40d40fa439c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:53:00.491029 26192 raft_consensus.cc:348] T 2883c249cea34a9bab50fa2a6ddbce96 P d565d39be5cb46d0b3a42bb91f89e557 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b84b24f6e6547adb247c40d40fa439c" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 32865 } } peers { permanent_uuid: "d565d39be5cb46d0b3a42bb91f89e557" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 38389 } } peers { permanent_uuid: "561728ee3b7b4ad5ba647c9f56d2bef2" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 32845 } }
I20260809 08:53:00.491148 26193 raft_consensus.cc:729] T 2883c249cea34a9bab50fa2a6ddbce96 P 3b84b24f6e6547adb247c40d40fa439c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b84b24f6e6547adb247c40d40fa439c, State: Initialized, Role: FOLLOWER
I20260809 08:53:00.491151 26192 raft_consensus.cc:374] T 2883c249cea34a9bab50fa2a6ddbce96 P d565d39be5cb46d0b3a42bb91f89e557 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:53:00.491151 26191 raft_consensus.cc:374] T 2883c249cea34a9bab50fa2a6ddbce96 P 561728ee3b7b4ad5ba647c9f56d2bef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:53:00.491189 26192 raft_consensus.cc:729] T 2883c249cea34a9bab50fa2a6ddbce96 P d565d39be5cb46d0b3a42bb91f89e557 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d565d39be5cb46d0b3a42bb91f89e557, State: Initialized, Role: FOLLOWER
I20260809 08:53:00.491189 26191 raft_consensus.cc:729] T 2883c249cea34a9bab50fa2a6ddbce96 P 561728ee3b7b4ad5ba647c9f56d2bef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 561728ee3b7b4ad5ba647c9f56d2bef2, State: Initialized, Role: FOLLOWER
I20260809 08:53:00.491287 26193 consensus_queue.cc:260] T 2883c249cea34a9bab50fa2a6ddbce96 P 3b84b24f6e6547adb247c40d40fa439c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b84b24f6e6547adb247c40d40fa439c" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 32865 } } peers { permanent_uuid: "d565d39be5cb46d0b3a42bb91f89e557" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 38389 } } peers { permanent_uuid: "561728ee3b7b4ad5ba647c9f56d2bef2" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 32845 } }
I20260809 08:53:00.491345 26192 consensus_queue.cc:260] T 2883c249cea34a9bab50fa2a6ddbce96 P d565d39be5cb46d0b3a42bb91f89e557 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b84b24f6e6547adb247c40d40fa439c" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 32865 } } peers { permanent_uuid: "d565d39be5cb46d0b3a42bb91f89e557" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 38389 } } peers { permanent_uuid: "561728ee3b7b4ad5ba647c9f56d2bef2" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 32845 } }
I20260809 08:53:00.491401 26191 consensus_queue.cc:260] T 2883c249cea34a9bab50fa2a6ddbce96 P 561728ee3b7b4ad5ba647c9f56d2bef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b84b24f6e6547adb247c40d40fa439c" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 32865 } } peers { permanent_uuid: "d565d39be5cb46d0b3a42bb91f89e557" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 38389 } } peers { permanent_uuid: "561728ee3b7b4ad5ba647c9f56d2bef2" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 32845 } }
I20260809 08:53:00.491580 26193 ts_tablet_manager.cc:1428] T 2883c249cea34a9bab50fa2a6ddbce96 P 3b84b24f6e6547adb247c40d40fa439c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 08:53:00.491648 26192 ts_tablet_manager.cc:1428] T 2883c249cea34a9bab50fa2a6ddbce96 P d565d39be5cb46d0b3a42bb91f89e557: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 08:53:00.491719 25956 heartbeater.cc:503] Master 127.8.154.62:41743 was elected leader, sending a full tablet report...
I20260809 08:53:00.491771 25754 heartbeater.cc:503] Master 127.8.154.62:41743 was elected leader, sending a full tablet report...
I20260809 08:53:00.491905 26158 heartbeater.cc:503] Master 127.8.154.62:41743 was elected leader, sending a full tablet report...
I20260809 08:53:00.491907 26191 ts_tablet_manager.cc:1428] T 2883c249cea34a9bab50fa2a6ddbce96 P 561728ee3b7b4ad5ba647c9f56d2bef2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:53:00.503571 26202 raft_consensus.cc:482] T 2883c249cea34a9bab50fa2a6ddbce96 P 3b84b24f6e6547adb247c40d40fa439c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:53:00.503710 26202 raft_consensus.cc:504] T 2883c249cea34a9bab50fa2a6ddbce96 P 3b84b24f6e6547adb247c40d40fa439c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b84b24f6e6547adb247c40d40fa439c" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 32865 } } peers { permanent_uuid: "d565d39be5cb46d0b3a42bb91f89e557" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 38389 } } peers { permanent_uuid: "561728ee3b7b4ad5ba647c9f56d2bef2" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 32845 } }
I20260809 08:53:00.504165 26202 leader_election.cc:288] T 2883c249cea34a9bab50fa2a6ddbce96 P 3b84b24f6e6547adb247c40d40fa439c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d565d39be5cb46d0b3a42bb91f89e557 (127.8.154.1:38389), 561728ee3b7b4ad5ba647c9f56d2bef2 (127.8.154.3:32845)
I20260809 08:53:00.507138 25958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:00.504344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:38389 (local address 127.8.154.2:45329)
0809 08:53:00.504394 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 08:53:00.504407 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:53:00.504452 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:53:00.504608 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:53:00.504609 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:53:00.504612 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:53:00.504691 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:53:00.504701 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:53:00.505478 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:53:00.505479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:53:00.506303 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:53:00.506305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:53:00.506876 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:53:00.506877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:53:00.506958 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:53:00.506962 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:53:00.507027 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 08:53:00.507066 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 08:53:00.507138 26179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:00.504401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:45329 (local address 127.8.154.1:38389)
0809 08:53:00.504451 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 08:53:00.504464 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 08:53:00.504479 (+ 15us) server_negotiation.cc:407] Connection header received
0809 08:53:00.504512 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:53:00.504513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:53:00.504526 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:53:00.504589 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:53:00.504780 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:00.505395 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:00.506386 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:00.506791 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:00.506876 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:53:00.506888 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 08:53:00.507032 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 08:53:00.507070 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 08:53:00.507413 26187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:00.504689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:42547 (local address 127.8.154.3:32845)
0809 08:53:00.504759 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 08:53:00.504761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:53:00.504773 (+ 12us) server_negotiation.cc:407] Connection header received
0809 08:53:00.504843 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:53:00.504844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:53:00.504855 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:53:00.504921 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:53:00.505090 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:00.505563 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:00.506611 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:00.507001 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:00.507082 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:53:00.507091 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 08:53:00.507297 (+ 206us) server_negotiation.cc:299] Negotiation successful
0809 08:53:00.507346 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 08:53:00.507426 26207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:00.504493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:32845 (local address 127.8.154.2:42547)
0809 08:53:00.504703 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 08:53:00.504732 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 08:53:00.504789 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:53:00.504925 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:53:00.504926 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:53:00.504927 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:53:00.504999 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:53:00.505001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:53:00.505656 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:53:00.505657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:53:00.506531 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:53:00.506533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:53:00.507084 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:53:00.507085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:53:00.507158 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:53:00.507162 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:53:00.507282 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 08:53:00.507332 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":3,"thread_start_us":130,"threads_started":1}
I20260809 08:53:00.507474 25679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2883c249cea34a9bab50fa2a6ddbce96" candidate_uuid: "3b84b24f6e6547adb247c40d40fa439c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d565d39be5cb46d0b3a42bb91f89e557" is_pre_election: true
I20260809 08:53:00.507668 25679 raft_consensus.cc:2454] T 2883c249cea34a9bab50fa2a6ddbce96 P d565d39be5cb46d0b3a42bb91f89e557 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b84b24f6e6547adb247c40d40fa439c in term 0.
I20260809 08:53:00.507925 25782 leader_election.cc:302] T 2883c249cea34a9bab50fa2a6ddbce96 P 3b84b24f6e6547adb247c40d40fa439c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b84b24f6e6547adb247c40d40fa439c, d565d39be5cb46d0b3a42bb91f89e557; no voters:
I20260809 08:53:00.508114 26202 raft_consensus.cc:2781] T 2883c249cea34a9bab50fa2a6ddbce96 P 3b84b24f6e6547adb247c40d40fa439c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:53:00.508024 26085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2883c249cea34a9bab50fa2a6ddbce96" candidate_uuid: "3b84b24f6e6547adb247c40d40fa439c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "561728ee3b7b4ad5ba647c9f56d2bef2" is_pre_election: true
I20260809 08:53:00.508170 26202 raft_consensus.cc:482] T 2883c249cea34a9bab50fa2a6ddbce96 P 3b84b24f6e6547adb247c40d40fa439c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:53:00.508193 26202 raft_consensus.cc:3037] T 2883c249cea34a9bab50fa2a6ddbce96 P 3b84b24f6e6547adb247c40d40fa439c [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:53:00.508245 26085 raft_consensus.cc:2454] T 2883c249cea34a9bab50fa2a6ddbce96 P 561728ee3b7b4ad5ba647c9f56d2bef2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3b84b24f6e6547adb247c40d40fa439c in term 0.
I20260809 08:53:00.509009 26202 raft_consensus.cc:504] T 2883c249cea34a9bab50fa2a6ddbce96 P 3b84b24f6e6547adb247c40d40fa439c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b84b24f6e6547adb247c40d40fa439c" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 32865 } } peers { permanent_uuid: "d565d39be5cb46d0b3a42bb91f89e557" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 38389 } } peers { permanent_uuid: "561728ee3b7b4ad5ba647c9f56d2bef2" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 32845 } }
I20260809 08:53:00.509117 26202 leader_election.cc:288] T 2883c249cea34a9bab50fa2a6ddbce96 P 3b84b24f6e6547adb247c40d40fa439c [CANDIDATE]: Term 1 election: Requested vote from peers d565d39be5cb46d0b3a42bb91f89e557 (127.8.154.1:38389), 561728ee3b7b4ad5ba647c9f56d2bef2 (127.8.154.3:32845)
I20260809 08:53:00.509338 26085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2883c249cea34a9bab50fa2a6ddbce96" candidate_uuid: "3b84b24f6e6547adb247c40d40fa439c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "561728ee3b7b4ad5ba647c9f56d2bef2"
I20260809 08:53:00.509443 26085 raft_consensus.cc:3037] T 2883c249cea34a9bab50fa2a6ddbce96 P 561728ee3b7b4ad5ba647c9f56d2bef2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:53:00.509325 25679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2883c249cea34a9bab50fa2a6ddbce96" candidate_uuid: "3b84b24f6e6547adb247c40d40fa439c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d565d39be5cb46d0b3a42bb91f89e557"
I20260809 08:53:00.509573 25679 raft_consensus.cc:3037] T 2883c249cea34a9bab50fa2a6ddbce96 P d565d39be5cb46d0b3a42bb91f89e557 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:53:00.510317 26085 raft_consensus.cc:2454] T 2883c249cea34a9bab50fa2a6ddbce96 P 561728ee3b7b4ad5ba647c9f56d2bef2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b84b24f6e6547adb247c40d40fa439c in term 1.
I20260809 08:53:00.510334 25679 raft_consensus.cc:2454] T 2883c249cea34a9bab50fa2a6ddbce96 P d565d39be5cb46d0b3a42bb91f89e557 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3b84b24f6e6547adb247c40d40fa439c in term 1.
I20260809 08:53:00.510521 25782 leader_election.cc:302] T 2883c249cea34a9bab50fa2a6ddbce96 P 3b84b24f6e6547adb247c40d40fa439c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b84b24f6e6547adb247c40d40fa439c, d565d39be5cb46d0b3a42bb91f89e557; no voters:
I20260809 08:53:00.510637 26202 raft_consensus.cc:2781] T 2883c249cea34a9bab50fa2a6ddbce96 P 3b84b24f6e6547adb247c40d40fa439c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:53:00.510814 26202 raft_consensus.cc:686] T 2883c249cea34a9bab50fa2a6ddbce96 P 3b84b24f6e6547adb247c40d40fa439c [term 1 LEADER]: Becoming Leader. State: Replica: 3b84b24f6e6547adb247c40d40fa439c, State: Running, Role: LEADER
I20260809 08:53:00.510922 26202 consensus_queue.cc:237] T 2883c249cea34a9bab50fa2a6ddbce96 P 3b84b24f6e6547adb247c40d40fa439c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b84b24f6e6547adb247c40d40fa439c" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 32865 } } peers { permanent_uuid: "d565d39be5cb46d0b3a42bb91f89e557" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 38389 } } peers { permanent_uuid: "561728ee3b7b4ad5ba647c9f56d2bef2" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 32845 } }
I20260809 08:53:00.511758 25466 catalog_manager.cc:5495] T 2883c249cea34a9bab50fa2a6ddbce96 P 3b84b24f6e6547adb247c40d40fa439c reported cstate change: term changed from 0 to 1, leader changed from <none> to 3b84b24f6e6547adb247c40d40fa439c (127.8.154.2). New cstate: current_term: 1 leader_uuid: "3b84b24f6e6547adb247c40d40fa439c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3b84b24f6e6547adb247c40d40fa439c" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 32865 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d565d39be5cb46d0b3a42bb91f89e557" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 38389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "561728ee3b7b4ad5ba647c9f56d2bef2" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 32845 } health_report { overall_health: UNKNOWN } } }
I20260809 08:53:00.520032 26184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:00.517603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38530 (local address 127.8.154.2:32865)
0809 08:53:00.517668 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 08:53:00.517669 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:53:00.517679 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:53:00.517756 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:53:00.517757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:53:00.517780 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:53:00.517863 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:53:00.518000 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:00.518477 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:00.519247 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:00.519450 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:00.519524 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:53:00.519682 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:53:00.519794 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:53:00.519877 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 08:53:00.519902 (+ 25us) server_negotiation.cc:299] Negotiation successful
0809 08:53:00.519925 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 08:53:00.521836 26085 raft_consensus.cc:1261] T 2883c249cea34a9bab50fa2a6ddbce96 P 561728ee3b7b4ad5ba647c9f56d2bef2 [term 1 FOLLOWER]: Refusing update from remote peer 3b84b24f6e6547adb247c40d40fa439c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:53:00.521886 25679 raft_consensus.cc:1261] T 2883c249cea34a9bab50fa2a6ddbce96 P d565d39be5cb46d0b3a42bb91f89e557 [term 1 FOLLOWER]: Refusing update from remote peer 3b84b24f6e6547adb247c40d40fa439c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:53:00.522207 26208 consensus_queue.cc:1035] T 2883c249cea34a9bab50fa2a6ddbce96 P 3b84b24f6e6547adb247c40d40fa439c [LEADER]: Connected to new peer: Peer: permanent_uuid: "561728ee3b7b4ad5ba647c9f56d2bef2" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 32845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:53:00.522346 26202 consensus_queue.cc:1035] T 2883c249cea34a9bab50fa2a6ddbce96 P 3b84b24f6e6547adb247c40d40fa439c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d565d39be5cb46d0b3a42bb91f89e557" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 38389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:53:00.524837 26212 mvcc.cc:205] Tried to move back new op lower bound from 7316543817814806528 to 7316543817773465600. Current Snapshot: MvccSnapshot[applied={T|T < 7316543817814806528}]
I20260809 08:53:00.526209 26214 mvcc.cc:205] Tried to move back new op lower bound from 7316543817814806528 to 7316543817773465600. Current Snapshot: MvccSnapshot[applied={T|T < 7316543817814806528}]
I20260809 08:53:00.527261 25466 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:53:00.527362 25466 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60190:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:53:00.528151 25466 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1a66cc87d1284a3fac21d5c90b7328a0: Sending DeleteTablet for 3 replicas of tablet 2883c249cea34a9bab50fa2a6ddbce96
I20260809 08:53:00.528468 25650 tablet_service.cc:1514] Processing DeleteTablet for tablet 2883c249cea34a9bab50fa2a6ddbce96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:53:00 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:54844
I20260809 08:53:00.528510 25860 tablet_service.cc:1514] Processing DeleteTablet for tablet 2883c249cea34a9bab50fa2a6ddbce96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:53:00 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:38518
I20260809 08:53:00.528527 26055 tablet_service.cc:1514] Processing DeleteTablet for tablet 2883c249cea34a9bab50fa2a6ddbce96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:53:00 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:51320
I20260809 08:53:00.528900 26225 tablet_replica.cc:330] T 2883c249cea34a9bab50fa2a6ddbce96 P d565d39be5cb46d0b3a42bb91f89e557: stopping tablet replica
I20260809 08:53:00.529006 26225 raft_consensus.cc:2229] T 2883c249cea34a9bab50fa2a6ddbce96 P d565d39be5cb46d0b3a42bb91f89e557 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:53:00.529052 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 25561
I20260809 08:53:00.529107 26225 raft_consensus.cc:2258] T 2883c249cea34a9bab50fa2a6ddbce96 P d565d39be5cb46d0b3a42bb91f89e557 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:53:00.529249 26228 tablet_replica.cc:330] T 2883c249cea34a9bab50fa2a6ddbce96 P 3b84b24f6e6547adb247c40d40fa439c: stopping tablet replica
I20260809 08:53:00.529326 26228 raft_consensus.cc:2229] T 2883c249cea34a9bab50fa2a6ddbce96 P 3b84b24f6e6547adb247c40d40fa439c [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:53:00.529400 26228 raft_consensus.cc:2258] T 2883c249cea34a9bab50fa2a6ddbce96 P 3b84b24f6e6547adb247c40d40fa439c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:53:00.529611 26232 tablet_replica.cc:330] T 2883c249cea34a9bab50fa2a6ddbce96 P 561728ee3b7b4ad5ba647c9f56d2bef2: stopping tablet replica
I20260809 08:53:00.529717 26232 raft_consensus.cc:2229] T 2883c249cea34a9bab50fa2a6ddbce96 P 561728ee3b7b4ad5ba647c9f56d2bef2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:53:00.529835 26232 raft_consensus.cc:2258] T 2883c249cea34a9bab50fa2a6ddbce96 P 561728ee3b7b4ad5ba647c9f56d2bef2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:53:00.533634 26228 ts_tablet_manager.cc:1905] T 2883c249cea34a9bab50fa2a6ddbce96 P 3b84b24f6e6547adb247c40d40fa439c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:53:00.535938 26232 ts_tablet_manager.cc:1905] T 2883c249cea34a9bab50fa2a6ddbce96 P 561728ee3b7b4ad5ba647c9f56d2bef2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:53:00.537577 26232 ts_tablet_manager.cc:1918] T 2883c249cea34a9bab50fa2a6ddbce96 P 561728ee3b7b4ad5ba647c9f56d2bef2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:53:00.537659 26232 log.cc:1196] T 2883c249cea34a9bab50fa2a6ddbce96 P 561728ee3b7b4ad5ba647c9f56d2bef2: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-2/wal/wals/2883c249cea34a9bab50fa2a6ddbce96
I20260809 08:53:00.537894 26232 ts_tablet_manager.cc:1939] T 2883c249cea34a9bab50fa2a6ddbce96 P 561728ee3b7b4ad5ba647c9f56d2bef2: Deleting consensus metadata
I20260809 08:53:00.538041 26228 ts_tablet_manager.cc:1918] T 2883c249cea34a9bab50fa2a6ddbce96 P 3b84b24f6e6547adb247c40d40fa439c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:53:00.538094 26228 log.cc:1196] T 2883c249cea34a9bab50fa2a6ddbce96 P 3b84b24f6e6547adb247c40d40fa439c: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265519942966-8808-0/minicluster-data/ts-1/wal/wals/2883c249cea34a9bab50fa2a6ddbce96
I20260809 08:53:00.538317 26228 ts_tablet_manager.cc:1939] T 2883c249cea34a9bab50fa2a6ddbce96 P 3b84b24f6e6547adb247c40d40fa439c: Deleting consensus metadata
I20260809 08:53:00.538715 25444 catalog_manager.cc:4841] TS 3b84b24f6e6547adb247c40d40fa439c (127.8.154.2:32865): tablet 2883c249cea34a9bab50fa2a6ddbce96 (table test-table [id=c40f04d70a21477a851a83045a991225]) successfully deleted
I20260809 08:53:00.539072 25441 catalog_manager.cc:4841] TS 561728ee3b7b4ad5ba647c9f56d2bef2 (127.8.154.3:32845): tablet 2883c249cea34a9bab50fa2a6ddbce96 (table test-table [id=c40f04d70a21477a851a83045a991225]) successfully deleted
W20260809 08:53:00.559446 25441 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.154.1:38389 (error 108)
I20260809 08:53:00.559690 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 25761
I20260809 08:53:00.562903 26178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:00.562730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:38389 (local address 127.0.0.1:54852)
0809 08:53:00.562808 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0809 08:53:00.562857 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.1:38389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 08:53:00.563035 25441 catalog_manager.cc:4584] TS d565d39be5cb46d0b3a42bb91f89e557 (127.8.154.1:38389): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2883c249cea34a9bab50fa2a6ddbce96: Network error: Client connection negotiation failed: client connection to 127.8.154.1:38389: connect: Connection refused (error 111)
I20260809 08:53:00.570441 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 25961
I20260809 08:53:00.587783 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 25418
2026-08-09T08:53:00Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2163 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:53:00 dist-test-slave-t3q3 krb5kdc[26238](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:53:00 dist-test-slave-t3q3 krb5kdc[26238](info): set up 2 sockets
Aug 09 08:53:00 dist-test-slave-t3q3 krb5kdc[26238](info): commencing operation
krb5kdc: starting...
W20260809 08:53:01.839912 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:53:01 dist-test-slave-t3q3 krb5kdc[26238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265581, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:53:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:53:01Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.8.154.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.8.154.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.8.154.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/rangeradmin_127.8.154.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.8.154.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/rangeradmin_127.8.154.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.8.154.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.8.154.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.8.154.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/rangerlookup_127.8.154.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.8.154.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/rangerlookup_127.8.154.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.154.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.8.154.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.154.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/HTTP_127.8.154.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.154.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/HTTP_127.8.154.20@KRBTEST.COM.keytab.
I20260809 08:53:02.006685 8808 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskBbavBz/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/postgres -l logfile start
2026-08-09 08:53:04.485 UTC [26281] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 08:53:04.485 UTC [26281] LOG: listening on IPv4 address "127.8.154.20", port 35669
2026-08-09 08:53:04.489 UTC [26281] LOG: listening on Unix socket "/tmp/.s.PGSQL.35669"
2026-08-09 08:53:04.510 UTC [26286] LOG: database system was shut down at 2026-08-09 08:53:03 UTC
2026-08-09 08:53:04.513 UTC [26281] LOG: database system is ready to accept connections
I20260809 08:53:05.680752 8808 mini_postgres.cc:96] Postgres bound to 35669
2026-08-09 08:53:05.689 UTC [26297] FATAL: database "slave" does not exist
127.8.154.20:35669 - accepting connections
I20260809 08:53:05.690388 8808 mini_ranger.cc:161] Starting Ranger...
I20260809 08:53:05.706352 8808 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 08:53:06.169282 8808 mini_ranger.cc:87] Created ranger Postgres database
I20260809 08:53:06.169389 8808 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ranger-admin
2026-08-09 08:53:06,552 [I] DB FLAVOR :POSTGRES
2026-08-09 08:53:06,553 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 08:53:06,553 [I] Checking connection..
2026-08-09 08:53:06,553 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:35669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 08:53:06,814 [I] Checking connection passed.
2026-08-09 08:53:06,815 [I] --------- Verifying version history table ---------
2026-08-09 08:53:06,815 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:35669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 08:53:07,099 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 08:53:07,100 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:35669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 08:53:07,367 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 08:53:07,368 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 08:53:07,368 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:35669/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 08:53:07.627 UTC [26393] WARNING: there is no transaction in progress
2026-08-09 08:53:07,636 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 08:53:07,637 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:35669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 08:53:07,891 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 08:53:07,891 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 08:53:07,891 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:35669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 08:53:08,126 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:35669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 08:53:08,393 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:35669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-t3q3.c.gcp-upstream.internal','N') ;"
2026-08-09 08:53:08,650 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:35669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 08:53:08,910 [I] Table x_portal_user does not exist in database ranger
2026-08-09 08:53:08,910 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:35669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 08:53:09,170 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 08:53:09,171 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:35669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 08:53:09,436 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:35669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 08:53:09,699 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 08:53:09,700 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:35669/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 08:53:10.621 UTC [26561] WARNING: there is no transaction in progress
2026-08-09 08:53:10.637 UTC [26561] WARNING: there is no transaction in progress
2026-08-09 08:53:10.653 UTC [26561] WARNING: there is no transaction in progress
2026-08-09 08:53:10.668 UTC [26561] WARNING: there is no transaction in progress
2026-08-09 08:53:10.684 UTC [26561] WARNING: there is no transaction in progress
2026-08-09 08:53:10.702 UTC [26561] WARNING: there is no transaction in progress
2026-08-09 08:53:10.760 UTC [26561] WARNING: there is no transaction in progress
2026-08-09 08:53:10.766 UTC [26561] WARNING: there is no transaction in progress
2026-08-09 08:53:10.780 UTC [26561] WARNING: there is no transaction in progress
2026-08-09 08:53:10.790 UTC [26561] WARNING: there is no transaction in progress
2026-08-09 08:53:10.799 UTC [26561] WARNING: there is no transaction in progress
2026-08-09 08:53:10.813 UTC [26561] WARNING: there is no transaction in progress
2026-08-09 08:53:10.823 UTC [26561] WARNING: there is no transaction in progress
2026-08-09 08:53:10.829 UTC [26561] WARNING: there is no transaction in progress
2026-08-09 08:53:10.836 UTC [26561] WARNING: there is no transaction in progress
2026-08-09 08:53:10.842 UTC [26561] WARNING: there is no transaction in progress
2026-08-09 08:53:11,455 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 08:53:11,456 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:35669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 08:53:11,712 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 08:53:11,712 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:35669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 08:53:11,943 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 08:53:11,944 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:35669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 08:53:12,193 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 08:53:12,193 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:35669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 08:53:12,459 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:35669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 08:53:12,719 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:35669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-t3q3.c.gcp-upstream.internal';"
2026-08-09 08:53:12,978 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 08:53:12,978 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:35669/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 08:53:13,228 [I] DB_PATCHES have already been applied
I20260809 08:53:13.234660 8808 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ranger-admin:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskBbavBz/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 08:53:13.234745 8808 mini_ranger.cc:193] Using host: 127.8.154.20
I20260809 08:53:13.238467 8808 mini_ranger.cc:227] Ranger admin URL: http://127.8.154.20:36293
Aug 09, 2026 8:53:13 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 8:53:13 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 8:53:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 8:53:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 8:53:13 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 8:53:14 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 8:53:14 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 08:53:14.640523 8808 mini_ranger.cc:160] Time spent starting Ranger: real 8.950s user 0.004s sys 0.007s
Aug 09, 2026 8:53:35 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 8:53:35 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 8:53:35 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 8:53:35 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 8:53:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 8:53:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 8:53:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 8:53:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 8:53:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 8:53:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 8:53:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 8:53:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 8:53:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 8:53:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 8:53:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 8:53:35 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 8:53:35 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 8:53:36 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 08:53:36.875332 8808 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.8.154.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.8.154.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.8.154.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/rangerkms_127.8.154.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.8.154.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/rangerkms_127.8.154.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.154.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.8.154.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.154.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/HTTP_127.8.154.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.154.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/HTTP_127.8.154.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 08:53:36 dist-test-slave-t3q3 krb5kdc[26238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265616, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 08:53:36.960584 8808 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 08:53:36.970885 8808 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 08:53:37.822151 8808 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 08:53:37.822248 8808 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ranger-kms
I20260809 08:53:37.822268 8808 mini_ranger_kms.cc:227] Using postgres at 127.8.154.20:35669
2026-08-09 08:53:38,089 [I] DB FLAVOR :POSTGRES
2026-08-09 08:53:38,090 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 08:53:38,090 [I] Checking connection
2026-08-09 08:53:38,090 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:35669/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 08:53:38,356 [I] connection success
2026-08-09 08:53:38,356 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 08:53:38,356 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 08:53:38,356 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:35669/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 08:53:38,640 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 08:53:38,640 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 08:53:38,641 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 08:53:38,641 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:35669/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 08:53:38,955 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 08:53:39.429916 8808 mini_ranger_kms.cc:310] Created kms service
I20260809 08:53:39.612421 8808 mini_ranger_kms.cc:326] Created kudu user
I20260809 08:53:39.657711 8808 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 08:53:39 dist-test-slave-t3q3 krb5kdc[26238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265616, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.8.154.20@KRBTEST.COM
I20260809 08:53:40.062323 8808 mini_ranger_kms.cc:384] Added ranger policy
I20260809 08:53:40.062440 8808 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ranger-kms:/tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 08:53:40.062462 8808 mini_ranger_kms.cc:240] Using host: 127.8.154.20
I20260809 08:53:40.066679 8808 mini_ranger_kms.cc:276] Ranger KMS PID: 26862
I20260809 08:53:40.066811 8808 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.8.154.20:47271
Aug 09, 2026 8:53:41 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 8:53:41 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 8:53:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 8:53:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 8:53:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 8:53:41 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 08:53:42 dist-test-slave-t3q3 krb5kdc[26238](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786265622, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.154.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:53:42.851449 8808 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.891s user 0.015s sys 0.232s
I20260809 08:53:42.851574 8808 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 08:53:42.851672 8808 mini_ranger_kms.cc:401] 127.8.154.20:47271/kms/v1/keys
Aug 09 08:53:42 dist-test-slave-t3q3 krb5kdc[26238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265622, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.154.20@KRBTEST.COM for HTTP/127.8.154.20@KRBTEST.COM
Aug 09 08:53:43 dist-test-slave-t3q3 krb5kdc[26238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265622, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.154.20@KRBTEST.COM for HTTP/127.8.154.20@KRBTEST.COM
Aug 09 08:53:45 dist-test-slave-t3q3 krb5kdc[26238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.8.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
I20260809 08:53:45.526445 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:45571
--webserver_interface=127.8.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:35789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:45571
--ranger_config_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.154.20:47271/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:53:45.635788 26934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:53:45.636003 26934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:53:45.636025 26934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:53:45.637214 26934 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:53:45.637260 26934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:53:45.637275 26934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:53:45.637287 26934 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:53:45.637300 26934 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:53:45.638937 26934 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:35789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/master-0/wal
--ranger_kms_url=127.8.154.20:47271/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:45571
--ranger_config_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:45571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26934
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:53:45.639192 26934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:53:45.639441 26934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:53:45.645856 26943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:53:45.646005 26942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:53:45.646046 26948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:53:45.645903 26934 server_base.cc:1034] running on GCE node
I20260809 08:53:45.646407 26934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:53:45.646721 26934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:53:45.647930 26934 hybrid_clock.cc:648] HybridClock initialized: now 1786265625647888 us; error 56 us; skew 500 ppm
Aug 09 08:53:45 dist-test-slave-t3q3 krb5kdc[26238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265625, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:53:45.654769 26934 init.cc:377] Logged in from keytab as kudu/127.8.154.62@KRBTEST.COM (short username kudu)
I20260809 08:53:45.655296 26934 webserver.cc:460] Webserver started at http://127.8.154.62:36877/ using document root <none> and password file <none>
I20260809 08:53:45.655531 26934 fs_manager.cc:359] Metadata directory not provided
I20260809 08:53:45.655601 26934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:53:45.655762 26934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 08:53:45 dist-test-slave-t3q3 krb5kdc[26238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265625, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for HTTP/127.8.154.20@KRBTEST.COM
I20260809 08:53:45.718011 26934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "537e4d16e6554f159dab102bf36579b3"
format_stamp: "Formatted at 2026-08-09 08:53:45 on dist-test-slave-t3q3"
server_key: "38aca7e648eb8b299649d52372049001"
server_key_iv: "76ab918f1e594b637569b923a846eaf2"
server_key_version: "kuduclusterkey@0"
I20260809 08:53:45.718532 26934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "537e4d16e6554f159dab102bf36579b3"
format_stamp: "Formatted at 2026-08-09 08:53:45 on dist-test-slave-t3q3"
server_key: "38aca7e648eb8b299649d52372049001"
server_key_iv: "76ab918f1e594b637569b923a846eaf2"
server_key_version: "kuduclusterkey@0"
I20260809 08:53:45.720170 26934 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260809 08:53:45.735020 26960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:53:45.735272 26934 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:53:45.735386 26934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "537e4d16e6554f159dab102bf36579b3"
format_stamp: "Formatted at 2026-08-09 08:53:45 on dist-test-slave-t3q3"
server_key: "38aca7e648eb8b299649d52372049001"
server_key_iv: "76ab918f1e594b637569b923a846eaf2"
server_key_version: "kuduclusterkey@0"
I20260809 08:53:45.735515 26934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:53:45.745169 26934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:53:45.747109 26934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:53:45.747288 26934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:53:45.751935 26934 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:45571
I20260809 08:53:45.751982 27048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:45571 every 8 connection(s)
I20260809 08:53:45.752379 26934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:53:45.753224 27051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:53:45.754268 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 26934
I20260809 08:53:45.754391 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:53:45.754585 8808 external_mini_cluster.cc:1428] Setting key 12868dcc62c1a103bc63ff09582eba2b
I20260809 08:53:45.756913 27051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 537e4d16e6554f159dab102bf36579b3: Bootstrap starting.
I20260809 08:53:45.757660 27051 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 537e4d16e6554f159dab102bf36579b3: Neither blocks nor log segments found. Creating new log.
I20260809 08:53:45.757993 27051 log.cc:824] T 00000000000000000000000000000000 P 537e4d16e6554f159dab102bf36579b3: Log is configured to *not* fsync() on all Append() calls
I20260809 08:53:45.758810 27051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 537e4d16e6554f159dab102bf36579b3: No bootstrap required, opened a new log
I20260809 08:53:45.761302 27051 raft_consensus.cc:348] T 00000000000000000000000000000000 P 537e4d16e6554f159dab102bf36579b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "537e4d16e6554f159dab102bf36579b3" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45571 } }
I20260809 08:53:45.761482 27051 raft_consensus.cc:374] T 00000000000000000000000000000000 P 537e4d16e6554f159dab102bf36579b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:53:45.761518 27051 raft_consensus.cc:729] T 00000000000000000000000000000000 P 537e4d16e6554f159dab102bf36579b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 537e4d16e6554f159dab102bf36579b3, State: Initialized, Role: FOLLOWER
I20260809 08:53:45.761693 27051 consensus_queue.cc:260] T 00000000000000000000000000000000 P 537e4d16e6554f159dab102bf36579b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "537e4d16e6554f159dab102bf36579b3" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45571 } }
I20260809 08:53:45.761776 27051 raft_consensus.cc:388] T 00000000000000000000000000000000 P 537e4d16e6554f159dab102bf36579b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:53:45.761806 27051 raft_consensus.cc:482] T 00000000000000000000000000000000 P 537e4d16e6554f159dab102bf36579b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:53:45.761874 27051 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 537e4d16e6554f159dab102bf36579b3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:53:45.762554 27051 raft_consensus.cc:504] T 00000000000000000000000000000000 P 537e4d16e6554f159dab102bf36579b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "537e4d16e6554f159dab102bf36579b3" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45571 } }
I20260809 08:53:45.762673 27051 leader_election.cc:302] T 00000000000000000000000000000000 P 537e4d16e6554f159dab102bf36579b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 537e4d16e6554f159dab102bf36579b3; no voters:
Aug 09 08:53:45 dist-test-slave-t3q3 krb5kdc[26238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:53:45.762887 27051 leader_election.cc:288] T 00000000000000000000000000000000 P 537e4d16e6554f159dab102bf36579b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:53:45.763159 27051 sys_catalog.cc:564] T 00000000000000000000000000000000 P 537e4d16e6554f159dab102bf36579b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:53:45.762956 27057 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 537e4d16e6554f159dab102bf36579b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:53:45.764297 27057 raft_consensus.cc:686] T 00000000000000000000000000000000 P 537e4d16e6554f159dab102bf36579b3 [term 1 LEADER]: Becoming Leader. State: Replica: 537e4d16e6554f159dab102bf36579b3, State: Running, Role: LEADER
I20260809 08:53:45.764797 27057 consensus_queue.cc:237] T 00000000000000000000000000000000 P 537e4d16e6554f159dab102bf36579b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "537e4d16e6554f159dab102bf36579b3" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45571 } }
I20260809 08:53:45.765424 27058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 537e4d16e6554f159dab102bf36579b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "537e4d16e6554f159dab102bf36579b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "537e4d16e6554f159dab102bf36579b3" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45571 } } }
I20260809 08:53:45.765501 27058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 537e4d16e6554f159dab102bf36579b3 [sys.catalog]: This master's current role is: LEADER
I20260809 08:53:45.765646 27058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 537e4d16e6554f159dab102bf36579b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 537e4d16e6554f159dab102bf36579b3. Latest consensus state: current_term: 1 leader_uuid: "537e4d16e6554f159dab102bf36579b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "537e4d16e6554f159dab102bf36579b3" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45571 } } }
I20260809 08:53:45.765686 27058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 537e4d16e6554f159dab102bf36579b3 [sys.catalog]: This master's current role is: LEADER
I20260809 08:53:45.766552 27051 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 08:53:45.769227 27054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:45.755177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56758 (local address 127.8.154.62:45571)
0809 08:53:45.755356 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 08:53:45.755366 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:53:45.756309 (+ 943us) server_negotiation.cc:407] Connection header received
0809 08:53:45.757025 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:53:45.757037 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:53:45.757112 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:53:45.757288 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:53:45.758198 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:45.759282 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:45.760133 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:45.760624 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:45.760756 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:53:45.763400 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:53:45.763405 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:53:45.763406 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:53:45.763419 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:53:45.765521 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:53:45.765967 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:53:45.765971 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:53:45.765972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:53:45.766088 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:53:45.766374 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:53:45.766375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:53:45.766376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:53:45.766692 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:53:45.766845 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 08:53:45.767035 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 08:53:45.767164 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":93,"threads_started":1}
I20260809 08:53:45.771148 27087 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:53:45.771814 27087 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:53:45.774019 27087 catalog_manager.cc:1347] Generated new cluster ID: 05846ef1a45c408ba2b31a1786b180de
I20260809 08:53:45.774073 27087 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:53:45.792380 27087 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:53:45.793303 27087 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:53:45.802347 27087 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 537e4d16e6554f159dab102bf36579b3: Generated new TSK 0
I20260809 08:53:45.802847 27087 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab.
Entry for principal kudu/127.8.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab.
I20260809 08:53:46.527248 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.154.20:47271/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.1:0
--local_ip_for_outbound_sockets=127.8.154.1
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:45571
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:35789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:53:46.633662 27133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:53:46.633893 27133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:53:46.633935 27133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:53:46.635087 27133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:53:46.635151 27133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:53:46.635197 27133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.1
I20260809 08:53:46.636744 27133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:35789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.8.154.20:47271/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:45571
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.1
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:53:46.637009 27133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:53:46.637239 27133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:53:46.642429 27139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:53:46.642517 27142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:53:46.642693 27138 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:53:46.643287 27133 server_base.cc:1034] running on GCE node
I20260809 08:53:46.643502 27133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:53:46.643751 27133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:53:46.644937 27133 hybrid_clock.cc:648] HybridClock initialized: now 1786265626644917 us; error 35 us; skew 500 ppm
Aug 09 08:53:46 dist-test-slave-t3q3 krb5kdc[26238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265626, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:53:46.650772 27133 init.cc:377] Logged in from keytab as kudu/127.8.154.1@KRBTEST.COM (short username kudu)
I20260809 08:53:46.651305 27133 webserver.cc:460] Webserver started at http://127.8.154.1:42783/ using document root <none> and password file <none>
I20260809 08:53:46.651588 27133 fs_manager.cc:359] Metadata directory not provided
I20260809 08:53:46.651660 27133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:53:46.651798 27133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 08:53:46 dist-test-slave-t3q3 krb5kdc[26238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265626, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for HTTP/127.8.154.20@KRBTEST.COM
I20260809 08:53:46.668530 27133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-0/data/instance:
uuid: "0b65703fe1334a2eac767c0145e2c454"
format_stamp: "Formatted at 2026-08-09 08:53:46 on dist-test-slave-t3q3"
server_key: "4f83a85bfc18bc251c9a8039d78a425d"
server_key_iv: "566da869f0545a4cc736c8b5c85b43b0"
server_key_version: "kuduclusterkey@0"
I20260809 08:53:46.669088 27133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance:
uuid: "0b65703fe1334a2eac767c0145e2c454"
format_stamp: "Formatted at 2026-08-09 08:53:46 on dist-test-slave-t3q3"
server_key: "4f83a85bfc18bc251c9a8039d78a425d"
server_key_iv: "566da869f0545a4cc736c8b5c85b43b0"
server_key_version: "kuduclusterkey@0"
I20260809 08:53:46.672736 27133 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.001s
I20260809 08:53:46.693482 27163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:53:46.693765 27133 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:53:46.693873 27133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-0/wal
uuid: "0b65703fe1334a2eac767c0145e2c454"
format_stamp: "Formatted at 2026-08-09 08:53:46 on dist-test-slave-t3q3"
server_key: "4f83a85bfc18bc251c9a8039d78a425d"
server_key_iv: "566da869f0545a4cc736c8b5c85b43b0"
server_key_version: "kuduclusterkey@0"
I20260809 08:53:46.693974 27133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:53:46.711092 27133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:53:46.713389 27133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:53:46.713557 27133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:53:46.713780 27133 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:53:46.714241 27133 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:53:46.714284 27133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:53:46.714308 27133 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:53:46.714321 27133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:53:46.722430 27133 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.1:40357
I20260809 08:53:46.722862 27133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
I20260809 08:53:46.725451 27342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.1:40357 every 8 connection(s)
I20260809 08:53:46.732304 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 27133
I20260809 08:53:46.732432 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance
I20260809 08:53:46.732697 8808 external_mini_cluster.cc:1428] Setting key 65a98271d632960f36b0aa13fda06877
Aug 09 08:53:46 dist-test-slave-t3q3 krb5kdc[26238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265626, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:53:46.746115 27349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:46.725133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45571 (local address 127.8.154.1:45841)
0809 08:53:46.725646 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0809 08:53:46.725690 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 08:53:46.726434 (+ 744us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:53:46.726746 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:53:46.726751 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:53:46.727074 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:53:46.727424 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:53:46.727430 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:53:46.732959 (+ 5529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:53:46.732961 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:53:46.734368 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:53:46.734371 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:53:46.739179 (+ 4808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:53:46.739180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:53:46.739434 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:53:46.740514 (+ 1080us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:53:46.740525 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:53:46.742196 (+ 1671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:53:46.744371 (+ 2175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:53:46.744381 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:53:46.744384 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:53:46.744628 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:53:46.744892 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:53:46.744893 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:53:46.744893 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:53:46.745047 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:53:46.745386 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:53:46.745394 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:53:46.745583 (+ 189us) client_negotiation.cc:769] Sending connection context
0809 08:53:46.745698 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 08:53:46.745868 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":109,"threads_started":1}
I20260809 08:53:46.746748 27350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:46.725729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:45841 (local address 127.8.154.62:45571)
0809 08:53:46.726279 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0809 08:53:46.726282 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:53:46.726317 (+ 35us) server_negotiation.cc:407] Connection header received
0809 08:53:46.726562 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:53:46.726563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:53:46.726578 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:53:46.726674 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:53:46.727542 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:46.728516 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:46.738650 (+ 10134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:46.739008 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:46.739089 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:53:46.742308 (+ 3219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:53:46.742310 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:53:46.742310 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:53:46.742318 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:53:46.744295 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:53:46.744740 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:53:46.744741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:53:46.744742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:53:46.744812 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:53:46.745126 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:53:46.745127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:53:46.745127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:53:46.745291 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:53:46.745348 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 08:53:46.746536 (+ 1188us) server_negotiation.cc:299] Negotiation successful
0809 08:53:46.746638 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":130,"threads_started":1}
I20260809 08:53:46.747555 27344 heartbeater.cc:347] Connected to a master server at 127.8.154.62:45571
I20260809 08:53:46.747653 27344 heartbeater.cc:464] Registering TS with master...
I20260809 08:53:46.747855 27344 heartbeater.cc:511] Master 127.8.154.62:45571 requested a full tablet report, sending...
I20260809 08:53:46.748512 26996 ts_manager.cc:194] Registered new tserver with Master: 0b65703fe1334a2eac767c0145e2c454 (127.8.154.1:40357)
I20260809 08:53:46.749198 26996 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.1@KRBTEST.COM'} at 127.8.154.1:45841
WARNING: no policy specified for kudu/127.8.154.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab.
Entry for principal kudu/127.8.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab.
I20260809 08:53:46.793298 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.154.20:47271/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.2:0
--local_ip_for_outbound_sockets=127.8.154.2
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:45571
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:35789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 08:53:46 dist-test-slave-t3q3 krb5kdc[26238](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786265626, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 08:53:46.889812 27359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:53:46.890041 27359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:53:46.890082 27359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:53:46.891887 27359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:53:46.891983 27359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:53:46.892041 27359 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.2
I20260809 08:53:46.894410 27359 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:35789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.8.154.20:47271/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:45571
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27359
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.2
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:53:46.894837 27359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:53:46.895152 27359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:53:46.904038 27371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:53:46.904007 27367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:53:46.904021 27369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:53:46.905296 27359 server_base.cc:1034] running on GCE node
I20260809 08:53:46.905565 27359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:53:46.905905 27359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:53:46.907145 27359 hybrid_clock.cc:648] HybridClock initialized: now 1786265626907120 us; error 35 us; skew 500 ppm
Aug 09 08:53:46 dist-test-slave-t3q3 krb5kdc[26238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265626, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:53:46.910039 27359 init.cc:377] Logged in from keytab as kudu/127.8.154.2@KRBTEST.COM (short username kudu)
I20260809 08:53:46.910605 27359 webserver.cc:460] Webserver started at http://127.8.154.2:38431/ using document root <none> and password file <none>
I20260809 08:53:46.910854 27359 fs_manager.cc:359] Metadata directory not provided
I20260809 08:53:46.910905 27359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:53:46.911023 27359 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 08:53:46 dist-test-slave-t3q3 krb5kdc[26238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265626, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for HTTP/127.8.154.20@KRBTEST.COM
I20260809 08:53:46.924491 27359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-1/data/instance:
uuid: "a805a08391b147c39843042a1210ba02"
format_stamp: "Formatted at 2026-08-09 08:53:46 on dist-test-slave-t3q3"
server_key: "9d0e96f53bfc0a566f9675cc6fcc9e1a"
server_key_iv: "1a8b290620a3c7ec51718eda1dfe94c1"
server_key_version: "kuduclusterkey@0"
I20260809 08:53:46.925055 27359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance:
uuid: "a805a08391b147c39843042a1210ba02"
format_stamp: "Formatted at 2026-08-09 08:53:46 on dist-test-slave-t3q3"
server_key: "9d0e96f53bfc0a566f9675cc6fcc9e1a"
server_key_iv: "1a8b290620a3c7ec51718eda1dfe94c1"
server_key_version: "kuduclusterkey@0"
I20260809 08:53:46.928525 27359 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 08:53:46.950260 27387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:53:46.950524 27359 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.000s sys 0.001s
I20260809 08:53:46.950631 27359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-1/wal
uuid: "a805a08391b147c39843042a1210ba02"
format_stamp: "Formatted at 2026-08-09 08:53:46 on dist-test-slave-t3q3"
server_key: "9d0e96f53bfc0a566f9675cc6fcc9e1a"
server_key_iv: "1a8b290620a3c7ec51718eda1dfe94c1"
server_key_version: "kuduclusterkey@0"
I20260809 08:53:46.950737 27359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:53:46.963047 27359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:53:46.965606 27359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:53:46.965816 27359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:53:46.966158 27359 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:53:46.966562 27359 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:53:46.966646 27359 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:53:46.966725 27359 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:53:46.966784 27359 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:53:46.974211 27359 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.2:37579
I20260809 08:53:46.974691 27359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
I20260809 08:53:46.976912 27566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.2:37579 every 8 connection(s)
I20260809 08:53:46.981056 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 27359
I20260809 08:53:46.981192 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance
I20260809 08:53:46.981386 8808 external_mini_cluster.cc:1428] Setting key b724bcdf11d6207c45bc5fe645e6b430
Aug 09 08:53:46 dist-test-slave-t3q3 krb5kdc[26238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265626, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:53:46.994266 27350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:46.977837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:49201 (local address 127.8.154.62:45571)
0809 08:53:46.977900 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 08:53:46.977902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:53:46.980375 (+ 2473us) server_negotiation.cc:407] Connection header received
0809 08:53:46.981102 (+ 727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:53:46.981104 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:53:46.981117 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:53:46.981157 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:53:46.982037 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:46.982756 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:46.984340 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:46.984667 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:46.984710 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:53:46.987304 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:53:46.987305 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:53:46.987306 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:53:46.987313 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:53:46.991551 (+ 4238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:53:46.992091 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:53:46.992092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:53:46.992092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:53:46.992156 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:53:46.992438 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:53:46.992439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:53:46.992439 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:53:46.992577 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:53:46.992621 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 08:53:46.994037 (+ 1416us) server_negotiation.cc:299] Negotiation successful
0809 08:53:46.994159 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 08:53:46.994400 27569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:46.977567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45571 (local address 127.8.154.2:49201)
0809 08:53:46.980232 (+ 2665us) negotiation.cc:107] Waiting for socket to connect
0809 08:53:46.980280 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 08:53:46.980959 (+ 679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:53:46.981361 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:53:46.981366 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:53:46.981644 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:53:46.981927 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:53:46.981933 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:53:46.982883 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:53:46.982883 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:53:46.984243 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:53:46.984246 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:53:46.984804 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:53:46.984805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:53:46.984947 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:53:46.985544 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:53:46.985554 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:53:46.987189 (+ 1635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:53:46.991667 (+ 4478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:53:46.991674 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:53:46.991676 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:53:46.991967 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:53:46.992233 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:53:46.992234 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:53:46.992235 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:53:46.992364 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:53:46.992685 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:53:46.992695 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:53:46.993598 (+ 903us) client_negotiation.cc:769] Sending connection context
0809 08:53:46.993974 (+ 376us) client_negotiation.cc:240] Negotiation successful
0809 08:53:46.994158 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2620,"thread_start_us":87,"threads_started":1}
I20260809 08:53:46.995106 27567 heartbeater.cc:347] Connected to a master server at 127.8.154.62:45571
I20260809 08:53:46.995213 27567 heartbeater.cc:464] Registering TS with master...
I20260809 08:53:46.995400 27567 heartbeater.cc:511] Master 127.8.154.62:45571 requested a full tablet report, sending...
I20260809 08:53:46.995828 26996 ts_manager.cc:194] Registered new tserver with Master: a805a08391b147c39843042a1210ba02 (127.8.154.2:37579)
I20260809 08:53:46.996325 26996 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.2@KRBTEST.COM'} at 127.8.154.2:49201
WARNING: no policy specified for kudu/127.8.154.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab.
Entry for principal kudu/127.8.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab.
I20260809 08:53:47.025020 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.154.20:47271/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.3:0
--local_ip_for_outbound_sockets=127.8.154.3
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:45571
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:35789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:53:47.171540 27573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:53:47.171876 27573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:53:47.171918 27573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:53:47.173874 27573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:53:47.173949 27573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:53:47.173995 27573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.3
I20260809 08:53:47.176326 27573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:35789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.8.154.20:47271/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:45571
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.3
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:53:47.176671 27573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:53:47.177048 27573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:53:47.183379 27579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:53:47.183333 27583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:53:47.183342 27580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:53:47.183779 27573 server_base.cc:1034] running on GCE node
I20260809 08:53:47.184104 27573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:53:47.184469 27573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:53:47.185683 27573 hybrid_clock.cc:648] HybridClock initialized: now 1786265627185662 us; error 43 us; skew 500 ppm
Aug 09 08:53:47 dist-test-slave-t3q3 krb5kdc[26238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265627, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:53:47.189430 27573 init.cc:377] Logged in from keytab as kudu/127.8.154.3@KRBTEST.COM (short username kudu)
I20260809 08:53:47.189963 27573 webserver.cc:460] Webserver started at http://127.8.154.3:46857/ using document root <none> and password file <none>
I20260809 08:53:47.190274 27573 fs_manager.cc:359] Metadata directory not provided
I20260809 08:53:47.190326 27573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:53:47.190461 27573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 08:53:47 dist-test-slave-t3q3 krb5kdc[26238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265627, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for HTTP/127.8.154.20@KRBTEST.COM
I20260809 08:53:47.209295 27573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-2/data/instance:
uuid: "3292afddf7a34c038f9e38e973d010e2"
format_stamp: "Formatted at 2026-08-09 08:53:47 on dist-test-slave-t3q3"
server_key: "a072fd4d99d269cb4907d4ee57e2ea8d"
server_key_iv: "882b6d91dce22aade86bab35c4f71746"
server_key_version: "kuduclusterkey@0"
I20260809 08:53:47.209970 27573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance:
uuid: "3292afddf7a34c038f9e38e973d010e2"
format_stamp: "Formatted at 2026-08-09 08:53:47 on dist-test-slave-t3q3"
server_key: "a072fd4d99d269cb4907d4ee57e2ea8d"
server_key_iv: "882b6d91dce22aade86bab35c4f71746"
server_key_version: "kuduclusterkey@0"
I20260809 08:53:47.216378 27573 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.000s
I20260809 08:53:47.235787 27596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:53:47.240193 27573 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.001s sys 0.000s
I20260809 08:53:47.240324 27573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-2/wal
uuid: "3292afddf7a34c038f9e38e973d010e2"
format_stamp: "Formatted at 2026-08-09 08:53:47 on dist-test-slave-t3q3"
server_key: "a072fd4d99d269cb4907d4ee57e2ea8d"
server_key_iv: "882b6d91dce22aade86bab35c4f71746"
server_key_version: "kuduclusterkey@0"
I20260809 08:53:47.240427 27573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:53:47.258350 27573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:53:47.260984 27573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:53:47.261231 27573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:53:47.261541 27573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:53:47.262053 27573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:53:47.262126 27573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:53:47.262203 27573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:53:47.262262 27573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:53:47.271075 27573 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.3:33167
I20260809 08:53:47.271555 27573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
I20260809 08:53:47.272420 27782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.3:33167 every 8 connection(s)
I20260809 08:53:47.276639 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 27573
I20260809 08:53:47.276757 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance
I20260809 08:53:47.276942 8808 external_mini_cluster.cc:1428] Setting key 8a58d767b3f843e1632dfec47dc8c0a7
Aug 09 08:53:47 dist-test-slave-t3q3 krb5kdc[26238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265627, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:53:47.286916 27350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:47.274075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:44909 (local address 127.8.154.62:45571)
0809 08:53:47.274126 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 08:53:47.274129 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:53:47.274516 (+ 387us) server_negotiation.cc:407] Connection header received
0809 08:53:47.275211 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:53:47.275213 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:53:47.275230 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:53:47.275277 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:53:47.276121 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:47.276976 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:47.280045 (+ 3069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:47.280403 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:47.280485 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:53:47.282897 (+ 2412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:53:47.282899 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:53:47.282899 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:53:47.282908 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:53:47.285059 (+ 2151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:53:47.285647 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:53:47.285648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:53:47.285649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:53:47.285718 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:53:47.286041 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:53:47.286042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:53:47.286042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:53:47.286203 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:53:47.286253 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 08:53:47.286706 (+ 453us) server_negotiation.cc:299] Negotiation successful
0809 08:53:47.286813 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 08:53:47.287122 27789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:47.273755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:45571 (local address 127.8.154.3:44909)
0809 08:53:47.274344 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0809 08:53:47.274414 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0809 08:53:47.275070 (+ 656us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:53:47.275376 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:53:47.275380 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:53:47.275671 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:53:47.275976 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:53:47.275984 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:53:47.277339 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:53:47.277340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:53:47.279921 (+ 2581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:53:47.279925 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:53:47.280496 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:53:47.280497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:53:47.280663 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:53:47.281330 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:53:47.281341 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:53:47.282785 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:53:47.285173 (+ 2388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:53:47.285182 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:53:47.285184 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:53:47.285539 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:53:47.285803 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:53:47.285803 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:53:47.285804 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:53:47.285949 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:53:47.286312 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:53:47.286319 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:53:47.286531 (+ 212us) client_negotiation.cc:769] Sending connection context
0809 08:53:47.286666 (+ 135us) client_negotiation.cc:240] Negotiation successful
0809 08:53:47.286837 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":105,"threads_started":1}
I20260809 08:53:47.287717 27785 heartbeater.cc:347] Connected to a master server at 127.8.154.62:45571
I20260809 08:53:47.287827 27785 heartbeater.cc:464] Registering TS with master...
I20260809 08:53:47.288144 27785 heartbeater.cc:511] Master 127.8.154.62:45571 requested a full tablet report, sending...
I20260809 08:53:47.288828 26996 ts_manager.cc:194] Registered new tserver with Master: 3292afddf7a34c038f9e38e973d010e2 (127.8.154.3:33167)
I20260809 08:53:47.289366 26996 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.3@KRBTEST.COM'} at 127.8.154.3:44909
I20260809 08:53:47.290762 8808 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:53:47.300659 27350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:47.292666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56766 (local address 127.8.154.62:45571)
0809 08:53:47.292711 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 08:53:47.292713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:53:47.292980 (+ 267us) server_negotiation.cc:407] Connection header received
0809 08:53:47.293106 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:53:47.293107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:53:47.293119 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:53:47.293183 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:53:47.293804 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:47.294343 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:47.295168 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:47.295398 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:47.295486 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:53:47.297107 (+ 1621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:53:47.297109 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:53:47.297109 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:53:47.297117 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:53:47.299105 (+ 1988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:53:47.299562 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:53:47.299563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:53:47.299564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:53:47.299636 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:53:47.299876 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:53:47.299877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:53:47.299877 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:53:47.300034 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:53:47.300171 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 08:53:47.300412 (+ 241us) server_negotiation.cc:299] Negotiation successful
0809 08:53:47.300543 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 08:53:47.303025 26996 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56766:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:53:47.305027 26996 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:53:47.314551 27806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:47.307811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:37579 (local address 127.0.0.1:46362)
0809 08:53:47.308959 (+ 1148us) negotiation.cc:107] Waiting for socket to connect
0809 08:53:47.308971 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:53:47.309017 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:53:47.309384 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:53:47.309385 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:53:47.309387 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:53:47.309425 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:53:47.309426 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:53:47.311125 (+ 1699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:53:47.311127 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:53:47.312644 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:53:47.312647 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:53:47.314222 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:53:47.314223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:53:47.314310 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:53:47.314324 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 08:53:47.314394 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 08:53:47.314445 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1128,"mutex_wait_us":2,"thread_start_us":95,"threads_started":1}
I20260809 08:53:47.315308 27810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:47.308443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46362 (local address 127.8.154.2:37579)
0809 08:53:47.309150 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0809 08:53:47.309155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:53:47.309165 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:53:47.309211 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:53:47.309213 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:53:47.309229 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:53:47.309306 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:53:47.309535 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:47.311025 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:47.312792 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:47.314029 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:47.314107 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:53:47.314185 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 08:53:47.315172 (+ 987us) server_negotiation.cc:299] Negotiation successful
0809 08:53:47.315219 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":104,"threads_started":1}
I20260809 08:53:47.316290 27805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:47.307811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:33167 (local address 127.0.0.1:58322)
0809 08:53:47.308501 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0809 08:53:47.308552 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 08:53:47.308632 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:53:47.308992 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:53:47.308996 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:53:47.308998 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:53:47.309123 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:53:47.309125 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:53:47.313570 (+ 4445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:53:47.313571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:53:47.314739 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:53:47.314742 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:53:47.315947 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:53:47.315948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:53:47.316032 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:53:47.316037 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:53:47.316156 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 08:53:47.316202 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":98,"threads_started":1}
I20260809 08:53:47.316350 27456 tablet_service.cc:1467] Processing CreateTablet for tablet 2c5435437d8e4b7e9da4af667bf862e1 (DEFAULT_TABLE table=test-table [id=6abf4b0c65ec403980c9524cff38573c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:53:47.316672 27808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:47.308140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58322 (local address 127.8.154.3:33167)
0809 08:53:47.308427 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0809 08:53:47.308432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:53:47.308734 (+ 302us) server_negotiation.cc:407] Connection header received
0809 08:53:47.308784 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:53:47.308787 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:53:47.308807 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:53:47.308885 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:53:47.309219 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:47.313468 (+ 4249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:47.314859 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:47.315744 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:47.315822 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:53:47.315907 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 08:53:47.316535 (+ 628us) server_negotiation.cc:299] Negotiation successful
0809 08:53:47.316585 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":94,"threads_started":1}
I20260809 08:53:47.316813 27456 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c5435437d8e4b7e9da4af667bf862e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:53:47.317451 27641 tablet_service.cc:1467] Processing CreateTablet for tablet 2c5435437d8e4b7e9da4af667bf862e1 (DEFAULT_TABLE table=test-table [id=6abf4b0c65ec403980c9524cff38573c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:53:47.317826 27641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c5435437d8e4b7e9da4af667bf862e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:53:47.321935 27813 tablet_bootstrap.cc:492] T 2c5435437d8e4b7e9da4af667bf862e1 P 3292afddf7a34c038f9e38e973d010e2: Bootstrap starting.
I20260809 08:53:47.321935 27811 tablet_bootstrap.cc:492] T 2c5435437d8e4b7e9da4af667bf862e1 P a805a08391b147c39843042a1210ba02: Bootstrap starting.
I20260809 08:53:47.322036 27806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:47.308084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:40357 (local address 127.0.0.1:54676)
0809 08:53:47.315001 (+ 6917us) negotiation.cc:107] Waiting for socket to connect
0809 08:53:47.315012 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:53:47.315047 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:53:47.316306 (+ 1259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:53:47.316307 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:53:47.316309 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:53:47.316350 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:53:47.316351 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:53:47.317631 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:53:47.317632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:53:47.318711 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:53:47.318714 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:53:47.321796 (+ 3082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:53:47.321797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:53:47.321860 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:53:47.321863 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:53:47.321918 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 08:53:47.321962 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6906,"thread_start_us":77,"threads_started":1}
I20260809 08:53:47.322547 27809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:47.308422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54676 (local address 127.8.154.1:40357)
0809 08:53:47.316001 (+ 7579us) server_negotiation.cc:206] Beginning negotiation
0809 08:53:47.316006 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:53:47.316022 (+ 16us) server_negotiation.cc:407] Connection header received
0809 08:53:47.316090 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:53:47.316092 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:53:47.316111 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:53:47.316221 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:53:47.316430 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:47.317525 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:47.320822 (+ 3297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:47.321622 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:47.321699 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:53:47.321758 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 08:53:47.322418 (+ 660us) server_negotiation.cc:299] Negotiation successful
0809 08:53:47.322468 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7536,"thread_start_us":65,"threads_started":1}
I20260809 08:53:47.323215 27811 tablet_bootstrap.cc:654] T 2c5435437d8e4b7e9da4af667bf862e1 P a805a08391b147c39843042a1210ba02: Neither blocks nor log segments found. Creating new log.
I20260809 08:53:47.323386 27238 tablet_service.cc:1467] Processing CreateTablet for tablet 2c5435437d8e4b7e9da4af667bf862e1 (DEFAULT_TABLE table=test-table [id=6abf4b0c65ec403980c9524cff38573c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:53:47.323776 27238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c5435437d8e4b7e9da4af667bf862e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:53:47.324031 27811 log.cc:824] T 2c5435437d8e4b7e9da4af667bf862e1 P a805a08391b147c39843042a1210ba02: Log is configured to *not* fsync() on all Append() calls
I20260809 08:53:47.325238 27811 tablet_bootstrap.cc:492] T 2c5435437d8e4b7e9da4af667bf862e1 P a805a08391b147c39843042a1210ba02: No bootstrap required, opened a new log
I20260809 08:53:47.325314 27811 ts_tablet_manager.cc:1397] T 2c5435437d8e4b7e9da4af667bf862e1 P a805a08391b147c39843042a1210ba02: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:53:47.327868 27811 raft_consensus.cc:348] T 2c5435437d8e4b7e9da4af667bf862e1 P a805a08391b147c39843042a1210ba02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a805a08391b147c39843042a1210ba02" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37579 } } peers { permanent_uuid: "0b65703fe1334a2eac767c0145e2c454" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40357 } } peers { permanent_uuid: "3292afddf7a34c038f9e38e973d010e2" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 33167 } }
I20260809 08:53:47.328094 27811 raft_consensus.cc:374] T 2c5435437d8e4b7e9da4af667bf862e1 P a805a08391b147c39843042a1210ba02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:53:47.328138 27811 raft_consensus.cc:729] T 2c5435437d8e4b7e9da4af667bf862e1 P a805a08391b147c39843042a1210ba02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a805a08391b147c39843042a1210ba02, State: Initialized, Role: FOLLOWER
I20260809 08:53:47.328562 27811 consensus_queue.cc:260] T 2c5435437d8e4b7e9da4af667bf862e1 P a805a08391b147c39843042a1210ba02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a805a08391b147c39843042a1210ba02" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37579 } } peers { permanent_uuid: "0b65703fe1334a2eac767c0145e2c454" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40357 } } peers { permanent_uuid: "3292afddf7a34c038f9e38e973d010e2" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 33167 } }
I20260809 08:53:47.328987 27811 ts_tablet_manager.cc:1428] T 2c5435437d8e4b7e9da4af667bf862e1 P a805a08391b147c39843042a1210ba02: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 08:53:47.329174 27567 heartbeater.cc:503] Master 127.8.154.62:45571 was elected leader, sending a full tablet report...
I20260809 08:53:47.329871 27813 tablet_bootstrap.cc:654] T 2c5435437d8e4b7e9da4af667bf862e1 P 3292afddf7a34c038f9e38e973d010e2: Neither blocks nor log segments found. Creating new log.
I20260809 08:53:47.331163 27813 log.cc:824] T 2c5435437d8e4b7e9da4af667bf862e1 P 3292afddf7a34c038f9e38e973d010e2: Log is configured to *not* fsync() on all Append() calls
I20260809 08:53:47.331591 27819 tablet_bootstrap.cc:492] T 2c5435437d8e4b7e9da4af667bf862e1 P 0b65703fe1334a2eac767c0145e2c454: Bootstrap starting.
I20260809 08:53:47.332204 27813 tablet_bootstrap.cc:492] T 2c5435437d8e4b7e9da4af667bf862e1 P 3292afddf7a34c038f9e38e973d010e2: No bootstrap required, opened a new log
I20260809 08:53:47.332350 27813 ts_tablet_manager.cc:1397] T 2c5435437d8e4b7e9da4af667bf862e1 P 3292afddf7a34c038f9e38e973d010e2: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.000s
I20260809 08:53:47.332552 27819 tablet_bootstrap.cc:654] T 2c5435437d8e4b7e9da4af667bf862e1 P 0b65703fe1334a2eac767c0145e2c454: Neither blocks nor log segments found. Creating new log.
I20260809 08:53:47.332937 27819 log.cc:824] T 2c5435437d8e4b7e9da4af667bf862e1 P 0b65703fe1334a2eac767c0145e2c454: Log is configured to *not* fsync() on all Append() calls
I20260809 08:53:47.333884 27819 tablet_bootstrap.cc:492] T 2c5435437d8e4b7e9da4af667bf862e1 P 0b65703fe1334a2eac767c0145e2c454: No bootstrap required, opened a new log
I20260809 08:53:47.333956 27819 ts_tablet_manager.cc:1397] T 2c5435437d8e4b7e9da4af667bf862e1 P 0b65703fe1334a2eac767c0145e2c454: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 08:53:47.335457 27813 raft_consensus.cc:348] T 2c5435437d8e4b7e9da4af667bf862e1 P 3292afddf7a34c038f9e38e973d010e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a805a08391b147c39843042a1210ba02" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37579 } } peers { permanent_uuid: "0b65703fe1334a2eac767c0145e2c454" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40357 } } peers { permanent_uuid: "3292afddf7a34c038f9e38e973d010e2" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 33167 } }
I20260809 08:53:47.335664 27813 raft_consensus.cc:374] T 2c5435437d8e4b7e9da4af667bf862e1 P 3292afddf7a34c038f9e38e973d010e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:53:47.335709 27813 raft_consensus.cc:729] T 2c5435437d8e4b7e9da4af667bf862e1 P 3292afddf7a34c038f9e38e973d010e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3292afddf7a34c038f9e38e973d010e2, State: Initialized, Role: FOLLOWER
I20260809 08:53:47.335834 27813 consensus_queue.cc:260] T 2c5435437d8e4b7e9da4af667bf862e1 P 3292afddf7a34c038f9e38e973d010e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a805a08391b147c39843042a1210ba02" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37579 } } peers { permanent_uuid: "0b65703fe1334a2eac767c0145e2c454" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40357 } } peers { permanent_uuid: "3292afddf7a34c038f9e38e973d010e2" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 33167 } }
I20260809 08:53:47.336144 27813 ts_tablet_manager.cc:1428] T 2c5435437d8e4b7e9da4af667bf862e1 P 3292afddf7a34c038f9e38e973d010e2: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 08:53:47.336329 27785 heartbeater.cc:503] Master 127.8.154.62:45571 was elected leader, sending a full tablet report...
I20260809 08:53:47.337626 27819 raft_consensus.cc:348] T 2c5435437d8e4b7e9da4af667bf862e1 P 0b65703fe1334a2eac767c0145e2c454 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a805a08391b147c39843042a1210ba02" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37579 } } peers { permanent_uuid: "0b65703fe1334a2eac767c0145e2c454" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40357 } } peers { permanent_uuid: "3292afddf7a34c038f9e38e973d010e2" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 33167 } }
I20260809 08:53:47.337806 27819 raft_consensus.cc:374] T 2c5435437d8e4b7e9da4af667bf862e1 P 0b65703fe1334a2eac767c0145e2c454 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:53:47.337910 27819 raft_consensus.cc:729] T 2c5435437d8e4b7e9da4af667bf862e1 P 0b65703fe1334a2eac767c0145e2c454 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b65703fe1334a2eac767c0145e2c454, State: Initialized, Role: FOLLOWER
I20260809 08:53:47.338064 27819 consensus_queue.cc:260] T 2c5435437d8e4b7e9da4af667bf862e1 P 0b65703fe1334a2eac767c0145e2c454 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a805a08391b147c39843042a1210ba02" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37579 } } peers { permanent_uuid: "0b65703fe1334a2eac767c0145e2c454" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40357 } } peers { permanent_uuid: "3292afddf7a34c038f9e38e973d010e2" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 33167 } }
I20260809 08:53:47.338425 27344 heartbeater.cc:503] Master 127.8.154.62:45571 was elected leader, sending a full tablet report...
I20260809 08:53:47.338403 27819 ts_tablet_manager.cc:1428] T 2c5435437d8e4b7e9da4af667bf862e1 P 0b65703fe1334a2eac767c0145e2c454: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
Aug 09 08:53:47 dist-test-slave-t3q3 krb5kdc[26238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265626, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for HTTP/127.8.154.20@KRBTEST.COM
I20260809 08:53:47.445552 27827 raft_consensus.cc:482] T 2c5435437d8e4b7e9da4af667bf862e1 P 0b65703fe1334a2eac767c0145e2c454 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:53:47.445739 27827 raft_consensus.cc:504] T 2c5435437d8e4b7e9da4af667bf862e1 P 0b65703fe1334a2eac767c0145e2c454 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a805a08391b147c39843042a1210ba02" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37579 } } peers { permanent_uuid: "0b65703fe1334a2eac767c0145e2c454" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40357 } } peers { permanent_uuid: "3292afddf7a34c038f9e38e973d010e2" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 33167 } }
I20260809 08:53:47.446183 27827 leader_election.cc:288] T 2c5435437d8e4b7e9da4af667bf862e1 P 0b65703fe1334a2eac767c0145e2c454 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a805a08391b147c39843042a1210ba02 (127.8.154.2:37579), 3292afddf7a34c038f9e38e973d010e2 (127.8.154.3:33167)
I20260809 08:53:47.450659 27808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:47.446606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:56923 (local address 127.8.154.3:33167)
0809 08:53:47.446657 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 08:53:47.446659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:53:47.446849 (+ 190us) server_negotiation.cc:407] Connection header received
0809 08:53:47.446979 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:53:47.446980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:53:47.446992 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:53:47.447033 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:53:47.447302 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:47.448102 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:47.449695 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:47.450249 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:47.450289 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:53:47.450299 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 08:53:47.450541 (+ 242us) server_negotiation.cc:299] Negotiation successful
0809 08:53:47.450584 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 08:53:47.450659 27810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:47.446610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:36633 (local address 127.8.154.2:37579)
0809 08:53:47.446679 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 08:53:47.446682 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:53:47.446849 (+ 167us) server_negotiation.cc:407] Connection header received
0809 08:53:47.446976 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:53:47.446978 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:53:47.446991 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:53:47.447025 (+ 34us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:53:47.447302 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:47.448099 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:47.449695 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:47.450240 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:47.450289 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:53:47.450299 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 08:53:47.450555 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 08:53:47.450593 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 08:53:47.451052 27830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:47.446435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:33167 (local address 127.8.154.1:56923)
0809 08:53:47.446767 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0809 08:53:47.446780 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:53:47.446882 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:53:47.447103 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:53:47.447104 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:53:47.447106 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:53:47.447215 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:53:47.447218 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:53:47.448236 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:53:47.448237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:53:47.449579 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:53:47.449583 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:53:47.450366 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:53:47.450366 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:53:47.450448 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:53:47.450455 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 08:53:47.450899 (+ 444us) client_negotiation.cc:240] Negotiation successful
0809 08:53:47.450950 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"mutex_wait_us":87,"thread_start_us":67,"threads_started":1}
I20260809 08:53:47.451059 27828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:47.446349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:37579 (local address 127.8.154.1:36633)
0809 08:53:47.446767 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0809 08:53:47.446780 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:53:47.446882 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:53:47.447103 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:53:47.447104 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:53:47.447107 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:53:47.447218 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:53:47.447219 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:53:47.448236 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:53:47.448237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:53:47.449579 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:53:47.449583 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:53:47.450364 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:53:47.450365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:53:47.450448 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:53:47.450454 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 08:53:47.450942 (+ 488us) client_negotiation.cc:240] Negotiation successful
0809 08:53:47.450973 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"mutex_wait_us":45,"thread_start_us":124,"threads_started":1}
I20260809 08:53:47.451534 27672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c5435437d8e4b7e9da4af667bf862e1" candidate_uuid: "0b65703fe1334a2eac767c0145e2c454" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3292afddf7a34c038f9e38e973d010e2" is_pre_election: true
I20260809 08:53:47.451722 27672 raft_consensus.cc:2454] T 2c5435437d8e4b7e9da4af667bf862e1 P 3292afddf7a34c038f9e38e973d010e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b65703fe1334a2eac767c0145e2c454 in term 0.
I20260809 08:53:47.451872 27488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c5435437d8e4b7e9da4af667bf862e1" candidate_uuid: "0b65703fe1334a2eac767c0145e2c454" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a805a08391b147c39843042a1210ba02" is_pre_election: true
I20260809 08:53:47.451979 27168 leader_election.cc:302] T 2c5435437d8e4b7e9da4af667bf862e1 P 0b65703fe1334a2eac767c0145e2c454 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b65703fe1334a2eac767c0145e2c454, 3292afddf7a34c038f9e38e973d010e2; no voters:
I20260809 08:53:47.452090 27488 raft_consensus.cc:2454] T 2c5435437d8e4b7e9da4af667bf862e1 P a805a08391b147c39843042a1210ba02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b65703fe1334a2eac767c0145e2c454 in term 0.
I20260809 08:53:47.452131 27827 raft_consensus.cc:2781] T 2c5435437d8e4b7e9da4af667bf862e1 P 0b65703fe1334a2eac767c0145e2c454 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:53:47.452175 27827 raft_consensus.cc:482] T 2c5435437d8e4b7e9da4af667bf862e1 P 0b65703fe1334a2eac767c0145e2c454 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:53:47.452201 27827 raft_consensus.cc:3037] T 2c5435437d8e4b7e9da4af667bf862e1 P 0b65703fe1334a2eac767c0145e2c454 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:53:47.452955 27827 raft_consensus.cc:504] T 2c5435437d8e4b7e9da4af667bf862e1 P 0b65703fe1334a2eac767c0145e2c454 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a805a08391b147c39843042a1210ba02" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37579 } } peers { permanent_uuid: "0b65703fe1334a2eac767c0145e2c454" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40357 } } peers { permanent_uuid: "3292afddf7a34c038f9e38e973d010e2" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 33167 } }
I20260809 08:53:47.453114 27827 leader_election.cc:288] T 2c5435437d8e4b7e9da4af667bf862e1 P 0b65703fe1334a2eac767c0145e2c454 [CANDIDATE]: Term 1 election: Requested vote from peers a805a08391b147c39843042a1210ba02 (127.8.154.2:37579), 3292afddf7a34c038f9e38e973d010e2 (127.8.154.3:33167)
I20260809 08:53:47.453272 27488 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c5435437d8e4b7e9da4af667bf862e1" candidate_uuid: "0b65703fe1334a2eac767c0145e2c454" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a805a08391b147c39843042a1210ba02"
I20260809 08:53:47.453353 27488 raft_consensus.cc:3037] T 2c5435437d8e4b7e9da4af667bf862e1 P a805a08391b147c39843042a1210ba02 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:53:47.453366 27672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c5435437d8e4b7e9da4af667bf862e1" candidate_uuid: "0b65703fe1334a2eac767c0145e2c454" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3292afddf7a34c038f9e38e973d010e2"
I20260809 08:53:47.453450 27672 raft_consensus.cc:3037] T 2c5435437d8e4b7e9da4af667bf862e1 P 3292afddf7a34c038f9e38e973d010e2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:53:47.454064 27488 raft_consensus.cc:2454] T 2c5435437d8e4b7e9da4af667bf862e1 P a805a08391b147c39843042a1210ba02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b65703fe1334a2eac767c0145e2c454 in term 1.
I20260809 08:53:47.454237 27164 leader_election.cc:302] T 2c5435437d8e4b7e9da4af667bf862e1 P 0b65703fe1334a2eac767c0145e2c454 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b65703fe1334a2eac767c0145e2c454, a805a08391b147c39843042a1210ba02; no voters:
I20260809 08:53:47.454317 27672 raft_consensus.cc:2454] T 2c5435437d8e4b7e9da4af667bf862e1 P 3292afddf7a34c038f9e38e973d010e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b65703fe1334a2eac767c0145e2c454 in term 1.
I20260809 08:53:47.454336 27827 raft_consensus.cc:2781] T 2c5435437d8e4b7e9da4af667bf862e1 P 0b65703fe1334a2eac767c0145e2c454 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:53:47.454499 27827 raft_consensus.cc:686] T 2c5435437d8e4b7e9da4af667bf862e1 P 0b65703fe1334a2eac767c0145e2c454 [term 1 LEADER]: Becoming Leader. State: Replica: 0b65703fe1334a2eac767c0145e2c454, State: Running, Role: LEADER
I20260809 08:53:47.454613 27827 consensus_queue.cc:237] T 2c5435437d8e4b7e9da4af667bf862e1 P 0b65703fe1334a2eac767c0145e2c454 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a805a08391b147c39843042a1210ba02" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37579 } } peers { permanent_uuid: "0b65703fe1334a2eac767c0145e2c454" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40357 } } peers { permanent_uuid: "3292afddf7a34c038f9e38e973d010e2" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 33167 } }
I20260809 08:53:47.455564 26994 catalog_manager.cc:5495] T 2c5435437d8e4b7e9da4af667bf862e1 P 0b65703fe1334a2eac767c0145e2c454 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b65703fe1334a2eac767c0145e2c454 (127.8.154.1). New cstate: current_term: 1 leader_uuid: "0b65703fe1334a2eac767c0145e2c454" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a805a08391b147c39843042a1210ba02" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37579 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b65703fe1334a2eac767c0145e2c454" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 40357 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3292afddf7a34c038f9e38e973d010e2" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 33167 } health_report { overall_health: UNKNOWN } } }
I20260809 08:53:47.462103 27809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:47.459312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54688 (local address 127.8.154.1:40357)
0809 08:53:47.459371 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 08:53:47.459373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:53:47.459388 (+ 15us) server_negotiation.cc:407] Connection header received
0809 08:53:47.459489 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:53:47.459490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:53:47.459509 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:53:47.459591 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:53:47.459750 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:47.460352 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:47.461253 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:53:47.461461 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:53:47.461538 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:53:47.461721 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:53:47.461828 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:53:47.461915 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 08:53:47.461956 (+ 41us) server_negotiation.cc:299] Negotiation successful
0809 08:53:47.461978 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 08:53:47.463867 27672 raft_consensus.cc:1261] T 2c5435437d8e4b7e9da4af667bf862e1 P 3292afddf7a34c038f9e38e973d010e2 [term 1 FOLLOWER]: Refusing update from remote peer 0b65703fe1334a2eac767c0145e2c454: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:53:47.463945 27488 raft_consensus.cc:1261] T 2c5435437d8e4b7e9da4af667bf862e1 P a805a08391b147c39843042a1210ba02 [term 1 FOLLOWER]: Refusing update from remote peer 0b65703fe1334a2eac767c0145e2c454: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:53:47.464291 27831 consensus_queue.cc:1035] T 2c5435437d8e4b7e9da4af667bf862e1 P 0b65703fe1334a2eac767c0145e2c454 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3292afddf7a34c038f9e38e973d010e2" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 33167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:53:47.464455 27831 consensus_queue.cc:1035] T 2c5435437d8e4b7e9da4af667bf862e1 P 0b65703fe1334a2eac767c0145e2c454 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a805a08391b147c39843042a1210ba02" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 37579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:53:47.468482 27840 mvcc.cc:205] Tried to move back new op lower bound from 7316544010089488384 to 7316544010054823936. Current Snapshot: MvccSnapshot[applied={T|T < 7316544010089488384}]
I20260809 08:53:47.469949 27838 mvcc.cc:205] Tried to move back new op lower bound from 7316544010089488384 to 7316544010054823936. Current Snapshot: MvccSnapshot[applied={T|T < 7316544010089488384}]
I20260809 08:53:47.470052 26994 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:53:47.470132 26994 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:53:47.470955 26994 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 537e4d16e6554f159dab102bf36579b3: Sending DeleteTablet for 3 replicas of tablet 2c5435437d8e4b7e9da4af667bf862e1
I20260809 08:53:47.471354 27456 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c5435437d8e4b7e9da4af667bf862e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:53:47 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:46362
I20260809 08:53:47.471361 27455 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c5435437d8e4b7e9da4af667bf862e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:53:47 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:46362
I20260809 08:53:47.471550 27641 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c5435437d8e4b7e9da4af667bf862e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:53:47 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:58322
I20260809 08:53:47.472021 27238 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c5435437d8e4b7e9da4af667bf862e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:53:47 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:54676
I20260809 08:53:47.472132 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 27133
I20260809 08:53:47.483856 27857 tablet_replica.cc:330] T 2c5435437d8e4b7e9da4af667bf862e1 P a805a08391b147c39843042a1210ba02: stopping tablet replica
I20260809 08:53:47.483983 27857 raft_consensus.cc:2229] T 2c5435437d8e4b7e9da4af667bf862e1 P a805a08391b147c39843042a1210ba02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:53:47.484133 27857 raft_consensus.cc:2258] T 2c5435437d8e4b7e9da4af667bf862e1 P a805a08391b147c39843042a1210ba02 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 08:53:47.490917 26963 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.154.1:40357 (error 108)
I20260809 08:53:47.491209 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 27359
I20260809 08:53:47.496086 27806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:47.491466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:40357 (local address 127.0.0.1:54696)
0809 08:53:47.495945 (+ 4479us) negotiation.cc:107] Waiting for socket to connect
0809 08:53:47.496007 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.1:40357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4451}
W20260809 08:53:47.496255 26963 catalog_manager.cc:4584] TS 0b65703fe1334a2eac767c0145e2c454 (127.8.154.1:40357): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2c5435437d8e4b7e9da4af667bf862e1: Network error: Client connection negotiation failed: client connection to 127.8.154.1:40357: connect: Connection refused (error 111)
I20260809 08:53:47.502482 27858 tablet_replica.cc:330] T 2c5435437d8e4b7e9da4af667bf862e1 P 3292afddf7a34c038f9e38e973d010e2: stopping tablet replica
I20260809 08:53:47.502648 27858 raft_consensus.cc:2229] T 2c5435437d8e4b7e9da4af667bf862e1 P 3292afddf7a34c038f9e38e973d010e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:53:47.502796 27858 raft_consensus.cc:2258] T 2c5435437d8e4b7e9da4af667bf862e1 P 3292afddf7a34c038f9e38e973d010e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:53:47.503392 27858 ts_tablet_manager.cc:1905] T 2c5435437d8e4b7e9da4af667bf862e1 P 3292afddf7a34c038f9e38e973d010e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:53:47.505246 27858 ts_tablet_manager.cc:1918] T 2c5435437d8e4b7e9da4af667bf862e1 P 3292afddf7a34c038f9e38e973d010e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:53:47.505339 27858 log.cc:1196] T 2c5435437d8e4b7e9da4af667bf862e1 P 3292afddf7a34c038f9e38e973d010e2: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265519942966-8808-0/minicluster-data/ts-2/wal/wals/2c5435437d8e4b7e9da4af667bf862e1
I20260809 08:53:47.506173 27806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:47.506047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:37579 (local address 127.0.0.1:46364)
0809 08:53:47.506104 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 08:53:47.506133 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.2:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 08:53:47.506280 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 27573
I20260809 08:53:47.517899 27806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:47.506360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:37579 (local address 127.0.0.1:46366)
0809 08:53:47.517789 (+ 11429us) negotiation.cc:107] Waiting for socket to connect
0809 08:53:47.517845 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.2:37579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11397}
I20260809 08:53:47.522675 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 26934
I20260809 08:53:47.522704 27806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:53:47.522579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:33167 (local address 127.0.0.1:58330)
0809 08:53:47.522647 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 08:53:47.522673 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.3:33167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 08:53:47.538278 8808 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 08:53:47.883334 8808 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 08:53:47.883445 8808 mini_ranger.cc:66] Stopping Ranger...
I20260809 08:53:48.237864 8808 mini_ranger.cc:68] Stopped Ranger
2026-08-09 08:53:48.238 UTC [26281] LOG: received smart shutdown request
2026-08-09 08:53:48.241 UTC [26281] LOG: background worker "logical replication launcher" (PID 26293) exited with exit code 1
2026-08-09 08:53:48.242 UTC [26288] LOG: shutting down
2026-08-09 08:53:48.387 UTC [26281] LOG: database system is shut down
2026-08-09T08:53:48Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47902 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:53:48 dist-test-slave-t3q3 krb5kdc[27871](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:53:48 dist-test-slave-t3q3 krb5kdc[27871](info): set up 2 sockets
Aug 09 08:53:48 dist-test-slave-t3q3 krb5kdc[27871](info): commencing operation
krb5kdc: starting...
W20260809 08:53:49.738493 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.219s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:53:49 dist-test-slave-t3q3 krb5kdc[27871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265629, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:53:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:53:49Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.8.154.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.8.154.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.8.154.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/rangeradmin_127.8.154.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.8.154.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/rangeradmin_127.8.154.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.8.154.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.8.154.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.8.154.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/rangerlookup_127.8.154.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.8.154.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/rangerlookup_127.8.154.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.154.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.8.154.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.154.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/HTTP_127.8.154.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.154.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/HTTP_127.8.154.20@KRBTEST.COM.keytab.
I20260809 08:53:49.906643 8808 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskBbavBz/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/postgres -l logfile start
2026-08-09 08:53:52.283 UTC [27921] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 08:53:52.283 UTC [27921] LOG: listening on IPv4 address "127.8.154.20", port 50487
2026-08-09 08:53:52.286 UTC [27921] LOG: listening on Unix socket "/tmp/.s.PGSQL.50487"
2026-08-09 08:53:52.322 UTC [27926] LOG: database system was shut down at 2026-08-09 08:53:50 UTC
2026-08-09 08:53:52.326 UTC [27921] LOG: database system is ready to accept connections
I20260809 08:53:53.488019 8808 mini_postgres.cc:96] Postgres bound to 50487
2026-08-09 08:53:53.495 UTC [27938] FATAL: database "slave" does not exist
127.8.154.20:50487 - accepting connections
I20260809 08:53:53.495884 8808 mini_ranger.cc:161] Starting Ranger...
I20260809 08:53:53.511014 8808 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 08:53:53.964538 8808 mini_ranger.cc:87] Created ranger Postgres database
I20260809 08:53:53.964637 8808 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ranger-admin
2026-08-09 08:53:54,268 [I] DB FLAVOR :POSTGRES
2026-08-09 08:53:54,268 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 08:53:54,268 [I] Checking connection..
2026-08-09 08:53:54,269 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:50487/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 08:53:54,537 [I] Checking connection passed.
2026-08-09 08:53:54,537 [I] --------- Verifying version history table ---------
2026-08-09 08:53:54,537 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:50487/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 08:53:54,789 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 08:53:54,789 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:50487/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 08:53:55,060 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 08:53:55,060 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 08:53:55,060 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:50487/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 08:53:55.324 UTC [28046] WARNING: there is no transaction in progress
2026-08-09 08:53:55,334 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 08:53:55,334 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:50487/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 08:53:55,578 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 08:53:55,578 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 08:53:55,578 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:50487/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 08:53:55,843 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:50487/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 08:53:56,086 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:50487/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-t3q3.c.gcp-upstream.internal','N') ;"
2026-08-09 08:53:56,332 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:50487/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 08:53:56,591 [I] Table x_portal_user does not exist in database ranger
2026-08-09 08:53:56,591 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:50487/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 08:53:56,849 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 08:53:56,850 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:50487/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 08:53:57,115 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:50487/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 08:53:57,375 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 08:53:57,375 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:50487/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 08:53:58.313 UTC [28213] WARNING: there is no transaction in progress
2026-08-09 08:53:58.328 UTC [28213] WARNING: there is no transaction in progress
2026-08-09 08:53:58.345 UTC [28213] WARNING: there is no transaction in progress
2026-08-09 08:53:58.359 UTC [28213] WARNING: there is no transaction in progress
2026-08-09 08:53:58.377 UTC [28213] WARNING: there is no transaction in progress
2026-08-09 08:53:58.393 UTC [28213] WARNING: there is no transaction in progress
2026-08-09 08:53:58.446 UTC [28213] WARNING: there is no transaction in progress
2026-08-09 08:53:58.453 UTC [28213] WARNING: there is no transaction in progress
2026-08-09 08:53:58.466 UTC [28213] WARNING: there is no transaction in progress
2026-08-09 08:53:58.476 UTC [28213] WARNING: there is no transaction in progress
2026-08-09 08:53:58.486 UTC [28213] WARNING: there is no transaction in progress
2026-08-09 08:53:58.499 UTC [28213] WARNING: there is no transaction in progress
2026-08-09 08:53:58.509 UTC [28213] WARNING: there is no transaction in progress
2026-08-09 08:53:58.515 UTC [28213] WARNING: there is no transaction in progress
2026-08-09 08:53:58.522 UTC [28213] WARNING: there is no transaction in progress
2026-08-09 08:53:58.529 UTC [28213] WARNING: there is no transaction in progress
2026-08-09 08:53:59,139 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 08:53:59,139 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:50487/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 08:53:59,385 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 08:53:59,385 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:50487/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 08:53:59,637 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 08:53:59,637 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:50487/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 08:53:59,904 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 08:53:59,904 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:50487/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 08:54:00,166 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:50487/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 08:54:00,425 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:50487/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-t3q3.c.gcp-upstream.internal';"
2026-08-09 08:54:00,674 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 08:54:00,675 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:50487/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 08:54:00,912 [I] DB_PATCHES have already been applied
I20260809 08:54:00.918052 8808 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ranger-admin:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskBbavBz/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 08:54:00.918133 8808 mini_ranger.cc:193] Using host: 127.8.154.20
I20260809 08:54:00.922371 8808 mini_ranger.cc:227] Ranger admin URL: http://127.8.154.20:34669
Aug 09, 2026 8:54:01 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 8:54:01 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 8:54:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 8:54:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 8:54:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 8:54:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 8:54:01 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 08:54:02.314502 8808 mini_ranger.cc:160] Time spent starting Ranger: real 8.819s user 0.004s sys 0.006s
Aug 09, 2026 8:54:22 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 8:54:22 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 8:54:22 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 8:54:22 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 8:54:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 8:54:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 8:54:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 8:54:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 8:54:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 8:54:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 8:54:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 8:54:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 8:54:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 8:54:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 8:54:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 8:54:22 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 8:54:22 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 8:54:23 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 08:54:24.185599 8808 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.8.154.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.8.154.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.8.154.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/rangerkms_127.8.154.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.8.154.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/rangerkms_127.8.154.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.8.154.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.8.154.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.8.154.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/HTTP_127.8.154.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.8.154.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/HTTP_127.8.154.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 08:54:24 dist-test-slave-t3q3 krb5kdc[27871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265664, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 08:54:24.278896 8808 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 08:54:24.289269 8808 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 08:54:25.102017 8808 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 08:54:25.102113 8808 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ranger-kms
I20260809 08:54:25.102135 8808 mini_ranger_kms.cc:227] Using postgres at 127.8.154.20:50487
2026-08-09 08:54:25,294 [I] DB FLAVOR :POSTGRES
2026-08-09 08:54:25,295 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 08:54:25,295 [I] Checking connection
2026-08-09 08:54:25,295 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:50487/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 08:54:25,530 [I] connection success
2026-08-09 08:54:25,530 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 08:54:25,530 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 08:54:25,531 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:50487/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 08:54:25,770 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 08:54:25,770 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 08:54:25,770 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 08:54:25,770 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.8.154.20:50487/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 08:54:26,042 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 08:54:26.521425 8808 mini_ranger_kms.cc:310] Created kms service
I20260809 08:54:26.696050 8808 mini_ranger_kms.cc:326] Created kudu user
I20260809 08:54:26.746299 8808 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 08:54:26 dist-test-slave-t3q3 krb5kdc[27871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265664, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.8.154.20@KRBTEST.COM
I20260809 08:54:27.173368 8808 mini_ranger_kms.cc:384] Added ranger policy
I20260809 08:54:27.173591 8808 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ranger-kms:/tmp/dist-test-taskBbavBz/build/release/bin/postgresql.jar:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskBbavBz/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 08:54:27.173614 8808 mini_ranger_kms.cc:240] Using host: 127.8.154.20
I20260809 08:54:27.177928 8808 mini_ranger_kms.cc:276] Ranger KMS PID: 28499
I20260809 08:54:27.178017 8808 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.8.154.20:53291
Aug 09, 2026 8:54:28 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 8:54:28 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 8:54:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 8:54:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 8:54:28 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 8:54:28 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 08:54:29 dist-test-slave-t3q3 krb5kdc[27871](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786265669, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.154.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 08:54:29 dist-test-slave-t3q3 krb5kdc[27871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265669, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.154.20@KRBTEST.COM for HTTP/127.8.154.20@KRBTEST.COM
I20260809 08:54:29.966233 8808 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.687s user 0.010s sys 0.166s
I20260809 08:54:29.966344 8808 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 08:54:29.966405 8808 mini_ranger_kms.cc:401] 127.8.154.20:53291/kms/v1/keys
Aug 09 08:54:30 dist-test-slave-t3q3 krb5kdc[27871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265669, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.8.154.20@KRBTEST.COM for HTTP/127.8.154.20@KRBTEST.COM
Aug 09 08:54:32 dist-test-slave-t3q3 krb5kdc[27871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.8.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
I20260809 08:54:32.425060 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:36715
--webserver_interface=127.8.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:34033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:36715
--ranger_config_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.154.20:53291/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:54:32.510737 28582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:54:32.510934 28582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:54:32.510957 28582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:54:32.512102 28582 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:54:32.512144 28582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:54:32.512159 28582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:54:32.512171 28582 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:54:32.512183 28582 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:54:32.513891 28582 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:34033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/master-0/wal
--ranger_kms_url=127.8.154.20:53291/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:36715
--ranger_config_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:36715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:54:32.514140 28582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:54:32.514376 28582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 08:54:32.520855 28582 server_base.cc:1034] running on GCE node
W20260809 08:54:32.520797 28589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:32.520823 28588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:32.520937 28591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:54:32.521423 28582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:54:32.521760 28582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:54:32.522943 28582 hybrid_clock.cc:648] HybridClock initialized: now 1786265672522924 us; error 35 us; skew 500 ppm
Aug 09 08:54:32 dist-test-slave-t3q3 krb5kdc[27871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265672, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:54:32.528267 28582 init.cc:377] Logged in from keytab as kudu/127.8.154.62@KRBTEST.COM (short username kudu)
I20260809 08:54:32.528836 28582 webserver.cc:460] Webserver started at http://127.8.154.62:45893/ using document root <none> and password file <none>
I20260809 08:54:32.529145 28582 fs_manager.cc:359] Metadata directory not provided
I20260809 08:54:32.529189 28582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:54:32.529326 28582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 08:54:32 dist-test-slave-t3q3 krb5kdc[27871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265672, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for HTTP/127.8.154.20@KRBTEST.COM
I20260809 08:54:32.577244 28582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "827c858b4a3641c1b751f2981370df0f"
format_stamp: "Formatted at 2026-08-09 08:54:32 on dist-test-slave-t3q3"
server_key: "faae31b086cc94e8431221d8e521df86"
server_key_iv: "87410cfc8ab9f1f9c7476f848eed68e8"
server_key_version: "kuduclusterkey@0"
I20260809 08:54:32.577708 28582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "827c858b4a3641c1b751f2981370df0f"
format_stamp: "Formatted at 2026-08-09 08:54:32 on dist-test-slave-t3q3"
server_key: "faae31b086cc94e8431221d8e521df86"
server_key_iv: "87410cfc8ab9f1f9c7476f848eed68e8"
server_key_version: "kuduclusterkey@0"
I20260809 08:54:32.579396 28582 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:54:32.592167 28605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:32.592414 28582 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:54:32.592540 28582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "827c858b4a3641c1b751f2981370df0f"
format_stamp: "Formatted at 2026-08-09 08:54:32 on dist-test-slave-t3q3"
server_key: "faae31b086cc94e8431221d8e521df86"
server_key_iv: "87410cfc8ab9f1f9c7476f848eed68e8"
server_key_version: "kuduclusterkey@0"
I20260809 08:54:32.592617 28582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:54:32.604017 28582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:54:32.606225 28582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:54:32.606397 28582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:54:32.611635 28582 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:36715
I20260809 08:54:32.611716 28688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:36715 every 8 connection(s)
I20260809 08:54:32.612146 28582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:54:32.612895 28690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:54:32.615307 28690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f: Bootstrap starting.
I20260809 08:54:32.615943 28690 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f: Neither blocks nor log segments found. Creating new log.
I20260809 08:54:32.616288 28690 log.cc:824] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f: Log is configured to *not* fsync() on all Append() calls
I20260809 08:54:32.617014 28690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f: No bootstrap required, opened a new log
I20260809 08:54:32.619176 28690 raft_consensus.cc:348] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "827c858b4a3641c1b751f2981370df0f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 36715 } }
I20260809 08:54:32.619339 28690 raft_consensus.cc:374] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:54:32.619364 28690 raft_consensus.cc:729] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 827c858b4a3641c1b751f2981370df0f, State: Initialized, Role: FOLLOWER
I20260809 08:54:32.619518 28690 consensus_queue.cc:260] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "827c858b4a3641c1b751f2981370df0f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 36715 } }
I20260809 08:54:32.619593 28690 raft_consensus.cc:388] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:54:32.619633 28690 raft_consensus.cc:482] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:54:32.619683 28690 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:54:32.620362 28690 raft_consensus.cc:504] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "827c858b4a3641c1b751f2981370df0f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 36715 } }
I20260809 08:54:32.620478 28690 leader_election.cc:302] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 827c858b4a3641c1b751f2981370df0f; no voters:
I20260809 08:54:32.620666 28690 leader_election.cc:288] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:54:32.620723 28696 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:54:32.620920 28696 raft_consensus.cc:686] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f [term 1 LEADER]: Becoming Leader. State: Replica: 827c858b4a3641c1b751f2981370df0f, State: Running, Role: LEADER
I20260809 08:54:32.620941 28690 sys_catalog.cc:564] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:54:32.621018 28696 consensus_queue.cc:237] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "827c858b4a3641c1b751f2981370df0f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 36715 } }
I20260809 08:54:32.621388 28696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 827c858b4a3641c1b751f2981370df0f. Latest consensus state: current_term: 1 leader_uuid: "827c858b4a3641c1b751f2981370df0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "827c858b4a3641c1b751f2981370df0f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 36715 } } }
I20260809 08:54:32.621822 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 28582
I20260809 08:54:32.621929 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:54:32.622115 8808 external_mini_cluster.cc:1428] Setting key d0841b9aace6bec269380bf2cf0bf5ac
I20260809 08:54:32.621625 28696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f [sys.catalog]: This master's current role is: LEADER
I20260809 08:54:32.622561 28690 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 08:54:32.622705 28717 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:54:32.621562 28697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "827c858b4a3641c1b751f2981370df0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "827c858b4a3641c1b751f2981370df0f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 36715 } } }
I20260809 08:54:32.623018 28697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f [sys.catalog]: This master's current role is: LEADER
I20260809 08:54:32.623344 28717 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:54:32.626513 28717 catalog_manager.cc:1347] Generated new cluster ID: 83152b3f44d342a5a52b88a5a1113238
I20260809 08:54:32.626559 28717 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 08:54:32 dist-test-slave-t3q3 krb5kdc[27871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:54:32.639958 28723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:32.622883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44362 (local address 127.8.154.62:36715)
0809 08:54:32.623279 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:32.623289 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:32.623326 (+ 37us) server_negotiation.cc:407] Connection header received
0809 08:54:32.623447 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:32.623454 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:32.623533 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:32.623669 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:54:32.624627 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:32.626112 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:32.627252 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:32.627915 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:32.628031 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:32.634025 (+ 5994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:54:32.634031 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:54:32.634032 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:54:32.634159 (+ 127us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:54:32.637367 (+ 3208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:32.637807 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:32.637811 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:32.637811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:32.637941 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:32.638172 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:32.638173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:32.638173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:32.638583 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:54:32.638775 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:32.639317 (+ 542us) server_negotiation.cc:299] Negotiation successful
0809 08:54:32.639650 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":72,"threads_started":1}
I20260809 08:54:32.651283 28717 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:54:32.652169 28717 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:54:32.656957 28717 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f: Generated new TSK 0
I20260809 08:54:32.657207 28717 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab.
Entry for principal kudu/127.8.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab.
I20260809 08:54:33.334654 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.154.20:53291/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.1:0
--local_ip_for_outbound_sockets=127.8.154.1
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:36715
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:34033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:54:33.427243 28761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:54:33.427482 28761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:54:33.427525 28761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:54:33.428761 28761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:54:33.428824 28761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:54:33.428870 28761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.1
I20260809 08:54:33.430433 28761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:34033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.8.154.20:53291/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:36715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.1
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:54:33.430688 28761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:54:33.430917 28761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:54:33.437570 28769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:33.437681 28766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:33.437582 28767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:54:33.438614 28761 server_base.cc:1034] running on GCE node
I20260809 08:54:33.438828 28761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:54:33.439116 28761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:54:33.440289 28761 hybrid_clock.cc:648] HybridClock initialized: now 1786265673440230 us; error 71 us; skew 500 ppm
Aug 09 08:54:33 dist-test-slave-t3q3 krb5kdc[27871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265673, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:54:33.443428 28761 init.cc:377] Logged in from keytab as kudu/127.8.154.1@KRBTEST.COM (short username kudu)
I20260809 08:54:33.443881 28761 webserver.cc:460] Webserver started at http://127.8.154.1:42477/ using document root <none> and password file <none>
I20260809 08:54:33.444180 28761 fs_manager.cc:359] Metadata directory not provided
I20260809 08:54:33.444250 28761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:54:33.444404 28761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 08:54:33 dist-test-slave-t3q3 krb5kdc[27871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265673, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for HTTP/127.8.154.20@KRBTEST.COM
I20260809 08:54:33.462960 28761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-0/data/instance:
uuid: "4f1db1398004416390bec4da38bdc5a0"
format_stamp: "Formatted at 2026-08-09 08:54:33 on dist-test-slave-t3q3"
server_key: "1114fea840f2a3a3d6c2dc1fed9a927f"
server_key_iv: "ea2abeceacc1d3a91c5d2ff6d168fdc2"
server_key_version: "kuduclusterkey@0"
I20260809 08:54:33.463444 28761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance:
uuid: "4f1db1398004416390bec4da38bdc5a0"
format_stamp: "Formatted at 2026-08-09 08:54:33 on dist-test-slave-t3q3"
server_key: "1114fea840f2a3a3d6c2dc1fed9a927f"
server_key_iv: "ea2abeceacc1d3a91c5d2ff6d168fdc2"
server_key_version: "kuduclusterkey@0"
I20260809 08:54:33.472551 28761 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.000s sys 0.004s
I20260809 08:54:33.488217 28786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:33.489861 28761 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260809 08:54:33.490001 28761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-0/wal
uuid: "4f1db1398004416390bec4da38bdc5a0"
format_stamp: "Formatted at 2026-08-09 08:54:33 on dist-test-slave-t3q3"
server_key: "1114fea840f2a3a3d6c2dc1fed9a927f"
server_key_iv: "ea2abeceacc1d3a91c5d2ff6d168fdc2"
server_key_version: "kuduclusterkey@0"
I20260809 08:54:33.490119 28761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:54:33.520723 28761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:54:33.523386 28761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:54:33.523555 28761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:54:33.523839 28761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:54:33.524251 28761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:54:33.524305 28761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:33.524339 28761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:54:33.524364 28761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:33.531967 28761 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.1:34745
I20260809 08:54:33.532087 28956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.1:34745 every 8 connection(s)
I20260809 08:54:33.532526 28761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
I20260809 08:54:33.536865 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 28761
I20260809 08:54:33.536979 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance
I20260809 08:54:33.537192 8808 external_mini_cluster.cc:1428] Setting key 3b3ed4826ad88989fce8f635c7b0b855
Aug 09 08:54:33 dist-test-slave-t3q3 krb5kdc[27871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265673, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:54:33.550024 28960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:33.533324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:36715 (local address 127.8.154.1:58339)
0809 08:54:33.537089 (+ 3765us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:33.537143 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 08:54:33.537812 (+ 669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:54:33.538280 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:54:33.538284 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:54:33.538548 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:54:33.538852 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:33.538859 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:33.539899 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:33.539900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:33.541239 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:33.541242 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:33.541814 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:33.541816 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:33.541961 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:33.542606 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:54:33.542617 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:54:33.544369 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:54:33.547960 (+ 3591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:33.547970 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:33.547973 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:33.548355 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:33.548590 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:33.548591 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:33.548592 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:33.548746 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:33.549236 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:54:33.549245 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:54:33.549453 (+ 208us) client_negotiation.cc:769] Sending connection context
0809 08:54:33.549575 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 08:54:33.549752 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3698,"thread_start_us":102,"threads_started":1}
I20260809 08:54:33.550762 28961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:33.536875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:58339 (local address 127.8.154.62:36715)
0809 08:54:33.538015 (+ 1140us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:33.538017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:33.538025 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:54:33.538050 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:33.538051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:33.538064 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:33.538141 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:54:33.538981 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:33.539786 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:33.541343 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:33.541661 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:33.541730 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:33.544495 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:54:33.544498 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:54:33.544498 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:54:33.544507 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:54:33.547850 (+ 3343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:33.548447 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:33.548448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:33.548448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:33.548519 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:33.548820 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:33.548821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:33.548821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:33.549116 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:54:33.549202 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:33.550562 (+ 1360us) server_negotiation.cc:299] Negotiation successful
0809 08:54:33.550662 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1107,"thread_start_us":97,"threads_started":1}
I20260809 08:54:33.551560 28957 heartbeater.cc:347] Connected to a master server at 127.8.154.62:36715
I20260809 08:54:33.551668 28957 heartbeater.cc:464] Registering TS with master...
I20260809 08:54:33.551875 28957 heartbeater.cc:511] Master 127.8.154.62:36715 requested a full tablet report, sending...
I20260809 08:54:33.552552 28629 ts_manager.cc:194] Registered new tserver with Master: 4f1db1398004416390bec4da38bdc5a0 (127.8.154.1:34745)
I20260809 08:54:33.553515 28629 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.1@KRBTEST.COM'} at 127.8.154.1:58339
WARNING: no policy specified for kudu/127.8.154.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab.
Entry for principal kudu/127.8.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab.
I20260809 08:54:33.585578 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.154.20:53291/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.2:0
--local_ip_for_outbound_sockets=127.8.154.2
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:36715
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:34033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:54:33.697486 28967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:54:33.697737 28967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:54:33.697780 28967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:54:33.698974 28967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:54:33.699039 28967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:54:33.699085 28967 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.2
I20260809 08:54:33.700759 28967 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:34033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.8.154.20:53291/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:36715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28967
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.2
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:54:33.701035 28967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:54:33.701292 28967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:54:33.706692 28981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:33.706669 28984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:54:33.707168 28967 server_base.cc:1034] running on GCE node
W20260809 08:54:33.708189 28982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:54:33.708442 28967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:54:33.708722 28967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:54:33.709877 28967 hybrid_clock.cc:648] HybridClock initialized: now 1786265673709857 us; error 38 us; skew 500 ppm
Aug 09 08:54:33 dist-test-slave-t3q3 krb5kdc[27871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265673, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:54:33.712774 28967 init.cc:377] Logged in from keytab as kudu/127.8.154.2@KRBTEST.COM (short username kudu)
I20260809 08:54:33.713317 28967 webserver.cc:460] Webserver started at http://127.8.154.2:41325/ using document root <none> and password file <none>
I20260809 08:54:33.713613 28967 fs_manager.cc:359] Metadata directory not provided
I20260809 08:54:33.713676 28967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:54:33.713814 28967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 08:54:33 dist-test-slave-t3q3 krb5kdc[27871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265673, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for HTTP/127.8.154.20@KRBTEST.COM
I20260809 08:54:33.730778 28967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-1/data/instance:
uuid: "97f93d66119c41a8b37ff268557ac86f"
format_stamp: "Formatted at 2026-08-09 08:54:33 on dist-test-slave-t3q3"
server_key: "adbf5f717212e800e430a911970c422a"
server_key_iv: "cf4dff5d39b1655863d2e6df7cac9dcf"
server_key_version: "kuduclusterkey@0"
I20260809 08:54:33.731350 28967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance:
uuid: "97f93d66119c41a8b37ff268557ac86f"
format_stamp: "Formatted at 2026-08-09 08:54:33 on dist-test-slave-t3q3"
server_key: "adbf5f717212e800e430a911970c422a"
server_key_iv: "cf4dff5d39b1655863d2e6df7cac9dcf"
server_key_version: "kuduclusterkey@0"
Aug 09 08:54:33 dist-test-slave-t3q3 krb5kdc[27871](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786265673, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:54:33.736477 28967 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.001s
I20260809 08:54:33.749280 28997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:33.749531 28967 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260809 08:54:33.749647 28967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-1/wal
uuid: "97f93d66119c41a8b37ff268557ac86f"
format_stamp: "Formatted at 2026-08-09 08:54:33 on dist-test-slave-t3q3"
server_key: "adbf5f717212e800e430a911970c422a"
server_key_iv: "cf4dff5d39b1655863d2e6df7cac9dcf"
server_key_version: "kuduclusterkey@0"
I20260809 08:54:33.749739 28967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:54:33.765470 28967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:54:33.767845 28967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:54:33.768013 28967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:54:33.768307 28967 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:54:33.768703 28967 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:54:33.768756 28967 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:33.768826 28967 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:54:33.768857 28967 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:33.775981 28967 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.2:44455
I20260809 08:54:33.776309 29177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.2:44455 every 8 connection(s)
I20260809 08:54:33.776559 28967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
I20260809 08:54:33.776911 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 28967
I20260809 08:54:33.777036 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance
I20260809 08:54:33.777302 8808 external_mini_cluster.cc:1428] Setting key 8795755b5838c22ace1a833bbd266800
Aug 09 08:54:33 dist-test-slave-t3q3 krb5kdc[27871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265673, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:54:33.788249 28961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:33.778510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:55389 (local address 127.8.154.62:36715)
0809 08:54:33.778559 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:33.778562 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:33.778906 (+ 344us) server_negotiation.cc:407] Connection header received
0809 08:54:33.779770 (+ 864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:33.779771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:33.779783 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:33.779814 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:54:33.780595 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:33.781392 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:33.782626 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:33.782860 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:33.782943 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:33.785288 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:54:33.785289 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:54:33.785290 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:54:33.785298 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:54:33.786782 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:33.787205 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:33.787206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:33.787206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:33.787269 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:33.787532 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:33.787533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:33.787533 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:33.787660 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:54:33.787735 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:33.788034 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 08:54:33.788151 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 08:54:33.788460 29181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:33.778222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:36715 (local address 127.8.154.2:55389)
0809 08:54:33.778778 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:33.778824 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 08:54:33.779610 (+ 786us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:54:33.780012 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:54:33.780018 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:54:33.780285 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:54:33.780487 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:33.780491 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:33.781509 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:33.781510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:33.782507 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:33.782510 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:33.782962 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:33.782963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:33.783102 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:33.783652 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:54:33.783661 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:54:33.785191 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:54:33.786879 (+ 1688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:33.786886 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:33.786888 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:33.787127 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:33.787343 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:33.787344 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:33.787344 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:33.787460 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:33.787726 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:54:33.787734 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:54:33.787895 (+ 161us) client_negotiation.cc:769] Sending connection context
0809 08:54:33.788001 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 08:54:33.788199 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":97,"threads_started":1}
I20260809 08:54:33.788959 29179 heartbeater.cc:347] Connected to a master server at 127.8.154.62:36715
I20260809 08:54:33.789080 29179 heartbeater.cc:464] Registering TS with master...
I20260809 08:54:33.789294 29179 heartbeater.cc:511] Master 127.8.154.62:36715 requested a full tablet report, sending...
I20260809 08:54:33.789698 28629 ts_manager.cc:194] Registered new tserver with Master: 97f93d66119c41a8b37ff268557ac86f (127.8.154.2:44455)
I20260809 08:54:33.790151 28629 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.2@KRBTEST.COM'} at 127.8.154.2:55389
WARNING: no policy specified for kudu/127.8.154.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab.
Entry for principal kudu/127.8.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab.
I20260809 08:54:33.814586 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.154.20:53291/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.3:0
--local_ip_for_outbound_sockets=127.8.154.3
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:36715
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:34033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:54:33.935451 29186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:54:33.935838 29186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:54:33.935935 29186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:54:33.937749 29186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:54:33.937867 29186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:54:33.937955 29186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.3
I20260809 08:54:33.940271 29186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:34033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.8.154.20:53291/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:36715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.3
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:54:33.940773 29186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:54:33.941057 29186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:54:33.948277 29195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:54:33.948663 29186 server_base.cc:1034] running on GCE node
W20260809 08:54:33.948923 29197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:33.949159 29194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:54:33.949467 29186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:54:33.949777 29186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:54:33.950970 29186 hybrid_clock.cc:648] HybridClock initialized: now 1786265673950945 us; error 35 us; skew 500 ppm
Aug 09 08:54:33 dist-test-slave-t3q3 krb5kdc[27871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265673, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:54:33.954833 29186 init.cc:377] Logged in from keytab as kudu/127.8.154.3@KRBTEST.COM (short username kudu)
I20260809 08:54:33.955433 29186 webserver.cc:460] Webserver started at http://127.8.154.3:37821/ using document root <none> and password file <none>
I20260809 08:54:33.955754 29186 fs_manager.cc:359] Metadata directory not provided
I20260809 08:54:33.955829 29186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:54:33.955981 29186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 08:54:33 dist-test-slave-t3q3 krb5kdc[27871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265673, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for HTTP/127.8.154.20@KRBTEST.COM
I20260809 08:54:33.970270 29186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-2/data/instance:
uuid: "f2faa26ce0e94f2c999153e341b5f7fd"
format_stamp: "Formatted at 2026-08-09 08:54:33 on dist-test-slave-t3q3"
server_key: "800c132e7415f050d5c7e1ebcf25011b"
server_key_iv: "fc0b25a46ea8be1060190e45f704d39b"
server_key_version: "kuduclusterkey@0"
I20260809 08:54:33.970892 29186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance:
uuid: "f2faa26ce0e94f2c999153e341b5f7fd"
format_stamp: "Formatted at 2026-08-09 08:54:33 on dist-test-slave-t3q3"
server_key: "800c132e7415f050d5c7e1ebcf25011b"
server_key_iv: "fc0b25a46ea8be1060190e45f704d39b"
server_key_version: "kuduclusterkey@0"
I20260809 08:54:33.973038 29186 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:54:33.992197 29207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:33.993048 29186 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.000s sys 0.001s
I20260809 08:54:33.993187 29186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-2/wal
uuid: "f2faa26ce0e94f2c999153e341b5f7fd"
format_stamp: "Formatted at 2026-08-09 08:54:33 on dist-test-slave-t3q3"
server_key: "800c132e7415f050d5c7e1ebcf25011b"
server_key_iv: "fc0b25a46ea8be1060190e45f704d39b"
server_key_version: "kuduclusterkey@0"
I20260809 08:54:33.993305 29186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:54:34.005236 29186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:54:34.008024 29186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:54:34.008313 29186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:54:34.008684 29186 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:54:34.009210 29186 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:54:34.009320 29186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:34.009408 29186 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:54:34.009480 29186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:34.027146 29186 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.3:34349
I20260809 08:54:34.027345 29387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.3:34349 every 8 connection(s)
I20260809 08:54:34.027657 29186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
Aug 09 08:54:34 dist-test-slave-t3q3 krb5kdc[27871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265673, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:54:34.035691 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 29186
I20260809 08:54:34.035818 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance
I20260809 08:54:34.036041 8808 external_mini_cluster.cc:1428] Setting key aa2639045e3fda7affedcbc1e50f2b31
I20260809 08:54:34.039971 28961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:34.028565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:55815 (local address 127.8.154.62:36715)
0809 08:54:34.028617 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:34.028620 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:34.028928 (+ 308us) server_negotiation.cc:407] Connection header received
0809 08:54:34.029645 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:34.029647 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:34.029661 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:34.029694 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:54:34.030581 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:34.031365 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:34.032933 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:34.033280 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:34.033332 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:34.036372 (+ 3040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:54:34.036374 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:54:34.036374 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:54:34.036382 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:54:34.038342 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:34.038836 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:34.038837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:34.038838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:34.038898 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:34.039199 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:34.039200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:34.039200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:34.039341 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:54:34.039383 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:34.039772 (+ 389us) server_negotiation.cc:299] Negotiation successful
0809 08:54:34.039867 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 08:54:34.040611 29394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:34.028544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:36715 (local address 127.8.154.3:55815)
0809 08:54:34.028755 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:34.028808 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 08:54:34.029490 (+ 682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:54:34.029898 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:54:34.029903 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:54:34.030176 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:54:34.030461 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:34.030467 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:34.031494 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:34.031495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:34.032815 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:34.032818 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:34.033451 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:34.033452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:34.033603 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:34.034256 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:54:34.034266 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:54:34.036256 (+ 1990us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:54:34.038439 (+ 2183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:34.038447 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:34.038449 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:34.038744 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:34.038970 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:34.038970 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:34.038970 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:34.039117 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:34.039428 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:54:34.039434 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:54:34.039612 (+ 178us) client_negotiation.cc:769] Sending connection context
0809 08:54:34.040202 (+ 590us) client_negotiation.cc:240] Negotiation successful
0809 08:54:34.040374 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":111,"threads_started":1}
I20260809 08:54:34.041188 29391 heartbeater.cc:347] Connected to a master server at 127.8.154.62:36715
I20260809 08:54:34.041333 29391 heartbeater.cc:464] Registering TS with master...
I20260809 08:54:34.041574 29391 heartbeater.cc:511] Master 127.8.154.62:36715 requested a full tablet report, sending...
I20260809 08:54:34.042003 28629 ts_manager.cc:194] Registered new tserver with Master: f2faa26ce0e94f2c999153e341b5f7fd (127.8.154.3:34349)
I20260809 08:54:34.042507 28629 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.3@KRBTEST.COM'} at 127.8.154.3:55815
I20260809 08:54:34.048444 8808 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:54:34.049130 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 28761
I20260809 08:54:34.094192 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 28967
I20260809 08:54:34.103475 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 29186
I20260809 08:54:34.123169 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 28582
I20260809 08:54:34.140897 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:36715
--webserver_interface=127.8.154.62
--webserver_port=45893
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:34033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:36715
--ranger_config_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.154.20:53291/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.154.20:53291/kms with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:54:34.224958 29396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:54:34.225178 29396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:54:34.225200 29396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:54:34.226344 29396 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:54:34.226392 29396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:54:34.226408 29396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:54:34.226419 29396 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:54:34.226431 29396 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:54:34.228044 29396 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:34033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.154.20:53291/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:36715
--ranger_config_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:36715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=45893
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29396
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:54:34.228315 29396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:54:34.228535 29396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:54:34.233624 29405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:34.233615 29408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:54:34.233770 29396 server_base.cc:1034] running on GCE node
W20260809 08:54:34.233780 29413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:54:34.234117 29396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:54:34.234403 29396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:54:34.235545 29396 hybrid_clock.cc:648] HybridClock initialized: now 1786265674235533 us; error 30 us; skew 500 ppm
Aug 09 08:54:34 dist-test-slave-t3q3 krb5kdc[27871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265674, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:54:34.238083 29396 init.cc:377] Logged in from keytab as kudu/127.8.154.62@KRBTEST.COM (short username kudu)
I20260809 08:54:34.238504 29396 webserver.cc:460] Webserver started at http://127.8.154.62:45893/ using document root <none> and password file <none>
I20260809 08:54:34.238744 29396 fs_manager.cc:359] Metadata directory not provided
I20260809 08:54:34.238816 29396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 08:54:34 dist-test-slave-t3q3 krb5kdc[27871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265674, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for HTTP/127.8.154.20@KRBTEST.COM
I20260809 08:54:34.311259 29396 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:54:34.312034 29428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:34.312244 29396 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:54:34.312345 29396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "827c858b4a3641c1b751f2981370df0f"
format_stamp: "Formatted at 2026-08-09 08:54:32 on dist-test-slave-t3q3"
server_key: "faae31b086cc94e8431221d8e521df86"
server_key_iv: "87410cfc8ab9f1f9c7476f848eed68e8"
server_key_version: "kuduclusterkey@0"
I20260809 08:54:34.312718 29396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:54:34.340415 29396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:54:34.342192 29396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:54:34.342351 29396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:54:34.346792 29396 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:36715
I20260809 08:54:34.346843 29504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:36715 every 8 connection(s)
I20260809 08:54:34.347226 29396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:54:34.347497 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 29396
I20260809 08:54:34.347827 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.154.20:53291/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.1:34745
--local_ip_for_outbound_sockets=127.8.154.1
--tserver_master_addrs=127.8.154.62:36715
--webserver_port=42477
--webserver_interface=127.8.154.1
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:34033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.154.20:53291/kms with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:54:34.348662 29505 sys_catalog.cc:263] Verifying existing consensus state
I20260809 08:54:34.349318 29505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f: Bootstrap starting.
I20260809 08:54:34.360198 29505 log.cc:824] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f: Log is configured to *not* fsync() on all Append() calls
I20260809 08:54:34.362421 29505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 08:54:34.362743 29505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f: Bootstrap complete.
I20260809 08:54:34.366160 29505 raft_consensus.cc:348] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "827c858b4a3641c1b751f2981370df0f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 36715 } }
I20260809 08:54:34.366470 29505 raft_consensus.cc:729] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 827c858b4a3641c1b751f2981370df0f, State: Initialized, Role: FOLLOWER
I20260809 08:54:34.366634 29505 consensus_queue.cc:260] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "827c858b4a3641c1b751f2981370df0f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 36715 } }
I20260809 08:54:34.366698 29505 raft_consensus.cc:388] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:54:34.366734 29505 raft_consensus.cc:482] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:54:34.366778 29505 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f [term 1 FOLLOWER]: Advancing to term 2
I20260809 08:54:34.368145 29505 raft_consensus.cc:504] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "827c858b4a3641c1b751f2981370df0f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 36715 } }
I20260809 08:54:34.368256 29505 leader_election.cc:302] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 827c858b4a3641c1b751f2981370df0f; no voters:
I20260809 08:54:34.368419 29505 leader_election.cc:288] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 08:54:34.368496 29512 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f [term 2 FOLLOWER]: Leader election won for term 2
I20260809 08:54:34.368659 29505 sys_catalog.cc:564] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:54:34.368698 29512 raft_consensus.cc:686] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f [term 2 LEADER]: Becoming Leader. State: Replica: 827c858b4a3641c1b751f2981370df0f, State: Running, Role: LEADER
I20260809 08:54:34.368976 29512 consensus_queue.cc:237] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "827c858b4a3641c1b751f2981370df0f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 36715 } }
I20260809 08:54:34.369297 29513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "827c858b4a3641c1b751f2981370df0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "827c858b4a3641c1b751f2981370df0f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 36715 } } }
I20260809 08:54:34.369388 29513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f [sys.catalog]: This master's current role is: LEADER
I20260809 08:54:34.369289 29514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 827c858b4a3641c1b751f2981370df0f. Latest consensus state: current_term: 2 leader_uuid: "827c858b4a3641c1b751f2981370df0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "827c858b4a3641c1b751f2981370df0f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 36715 } } }
I20260809 08:54:34.369554 29514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f [sys.catalog]: This master's current role is: LEADER
I20260809 08:54:34.369949 29525 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:54:34.370496 29525 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:54:34.370631 29525 catalog_manager.cc:1259] Loaded cluster ID: 83152b3f44d342a5a52b88a5a1113238
I20260809 08:54:34.370720 29525 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:54:34.370918 29505 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 08:54:34.374894 29525 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:54:34.375494 29525 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f: Loaded TSK: 0
I20260809 08:54:34.375801 29525 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 08:54:34.466727 29509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:54:34.466948 29509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:54:34.466990 29509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:54:34.468583 29509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:54:34.468642 29509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:54:34.468688 29509 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.1
I20260809 08:54:34.470432 29509 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:34033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.154.20:53291/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.1:34745
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.154.1
--webserver_port=42477
--tserver_master_addrs=127.8.154.62:36715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.1
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:54:34.470692 29509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:54:34.470924 29509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:54:34.477097 29566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:34.477137 29567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:34.477149 29569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:54:34.484408 29509 server_base.cc:1034] running on GCE node
I20260809 08:54:34.484740 29509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:54:34.485078 29509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:54:34.486251 29509 hybrid_clock.cc:648] HybridClock initialized: now 1786265674486213 us; error 60 us; skew 500 ppm
Aug 09 08:54:34 dist-test-slave-t3q3 krb5kdc[27871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265674, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:54:34.489272 29509 init.cc:377] Logged in from keytab as kudu/127.8.154.1@KRBTEST.COM (short username kudu)
I20260809 08:54:34.489751 29509 webserver.cc:460] Webserver started at http://127.8.154.1:42477/ using document root <none> and password file <none>
I20260809 08:54:34.490016 29509 fs_manager.cc:359] Metadata directory not provided
I20260809 08:54:34.490108 29509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 08:54:34 dist-test-slave-t3q3 krb5kdc[27871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265674, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for HTTP/127.8.154.20@KRBTEST.COM
I20260809 08:54:34.510215 29509 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:54:34.511276 29586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:34.511518 29509 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:54:34.511624 29509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-0/wal
uuid: "4f1db1398004416390bec4da38bdc5a0"
format_stamp: "Formatted at 2026-08-09 08:54:33 on dist-test-slave-t3q3"
server_key: "1114fea840f2a3a3d6c2dc1fed9a927f"
server_key_iv: "ea2abeceacc1d3a91c5d2ff6d168fdc2"
server_key_version: "kuduclusterkey@0"
I20260809 08:54:34.512102 29509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:54:34.530329 29509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:54:34.533011 29509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:54:34.533260 29509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:54:34.533602 29509 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:54:34.534063 29509 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:54:34.534168 29509 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:34.534271 29509 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:54:34.534348 29509 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:34.541559 29509 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.1:34745
I20260809 08:54:34.542380 29509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
I20260809 08:54:34.542357 29765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.1:34745 every 8 connection(s)
I20260809 08:54:34.546721 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 29509
I20260809 08:54:34.547058 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.154.20:53291/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.2:44455
--local_ip_for_outbound_sockets=127.8.154.2
--tserver_master_addrs=127.8.154.62:36715
--webserver_port=41325
--webserver_interface=127.8.154.2
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:34033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.154.20:53291/kms with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 08:54:34 dist-test-slave-t3q3 krb5kdc[27871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265674, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:54:34.555836 29769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:34.543203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:36715 (local address 127.8.154.1:52155)
0809 08:54:34.543895 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:34.543945 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 08:54:34.544872 (+ 927us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:54:34.545527 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:54:34.545532 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:54:34.545801 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:54:34.546083 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:34.546090 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:34.547397 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:34.547398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:34.548321 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:34.548324 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:34.549163 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:34.549164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:34.549320 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:34.549930 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:54:34.549940 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:54:34.551671 (+ 1731us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:54:34.553889 (+ 2218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:34.553896 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:34.553898 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:34.554191 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:34.554488 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:34.554489 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:34.554489 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:34.554620 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:34.555137 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:54:34.555144 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:54:34.555300 (+ 156us) client_negotiation.cc:769] Sending connection context
0809 08:54:34.555413 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 08:54:34.555580 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":96,"threads_started":1}
I20260809 08:54:34.556640 29770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:34.543668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:52155 (local address 127.8.154.62:36715)
0809 08:54:34.544560 (+ 892us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:34.544570 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:34.544614 (+ 44us) server_negotiation.cc:407] Connection header received
0809 08:54:34.545166 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:34.545172 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:34.545232 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:34.545364 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:54:34.546220 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:34.547293 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:34.548421 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:34.549034 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:34.549165 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:34.551797 (+ 2632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:54:34.551802 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:54:34.551802 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:54:34.551815 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:54:34.553790 (+ 1975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:34.554288 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:34.554292 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:34.554292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:34.554408 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:34.554714 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:34.554715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:34.554715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:34.555018 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:54:34.555103 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:34.556243 (+ 1140us) server_negotiation.cc:299] Negotiation successful
0809 08:54:34.556389 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":836,"thread_start_us":85,"threads_started":1}
I20260809 08:54:34.557369 29767 heartbeater.cc:347] Connected to a master server at 127.8.154.62:36715
I20260809 08:54:34.557477 29767 heartbeater.cc:464] Registering TS with master...
I20260809 08:54:34.557679 29767 heartbeater.cc:511] Master 127.8.154.62:36715 requested a full tablet report, sending...
W20260809 08:54:34.558604 29767 heartbeater.cc:644] Failed to heartbeat to 127.8.154.62:36715 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 08:54:34.558689 29767 heartbeater.cc:464] Registering TS with master...
I20260809 08:54:34.558755 29767 heartbeater.cc:511] Master 127.8.154.62:36715 requested a full tablet report, sending...
I20260809 08:54:34.559172 29767 heartbeater.cc:464] Registering TS with master...
I20260809 08:54:34.559252 29767 heartbeater.cc:511] Master 127.8.154.62:36715 requested a full tablet report, sending...
W20260809 08:54:34.559615 29767 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 08:54:34.650154 29772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:54:34.650379 29772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:54:34.650421 29772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:54:34.651592 29772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:54:34.651659 29772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:54:34.651706 29772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.2
I20260809 08:54:34.653486 29772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:34033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.154.20:53291/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.2:44455
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.154.2
--webserver_port=41325
--tserver_master_addrs=127.8.154.62:36715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.2
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:54:34.653756 29772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:54:34.654000 29772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:54:34.659392 29782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:34.659636 29780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:54:34.659857 29772 server_base.cc:1034] running on GCE node
W20260809 08:54:34.659873 29779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:54:34.660254 29772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:54:34.660548 29772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:54:34.661733 29772 hybrid_clock.cc:648] HybridClock initialized: now 1786265674661716 us; error 29 us; skew 500 ppm
Aug 09 08:54:34 dist-test-slave-t3q3 krb5kdc[27871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265674, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:54:34.665107 29772 init.cc:377] Logged in from keytab as kudu/127.8.154.2@KRBTEST.COM (short username kudu)
I20260809 08:54:34.665728 29772 webserver.cc:460] Webserver started at http://127.8.154.2:41325/ using document root <none> and password file <none>
I20260809 08:54:34.665994 29772 fs_manager.cc:359] Metadata directory not provided
I20260809 08:54:34.666090 29772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 08:54:34 dist-test-slave-t3q3 krb5kdc[27871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265674, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for HTTP/127.8.154.20@KRBTEST.COM
I20260809 08:54:34.699827 29772 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.000s sys 0.001s
I20260809 08:54:34.703946 29794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:34.704234 29772 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.000s
I20260809 08:54:34.704344 29772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-1/wal
uuid: "97f93d66119c41a8b37ff268557ac86f"
format_stamp: "Formatted at 2026-08-09 08:54:33 on dist-test-slave-t3q3"
server_key: "adbf5f717212e800e430a911970c422a"
server_key_iv: "cf4dff5d39b1655863d2e6df7cac9dcf"
server_key_version: "kuduclusterkey@0"
I20260809 08:54:34.704852 29772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:54:34.715945 29772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:54:34.718451 29772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:54:34.718611 29772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:54:34.718871 29772 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:54:34.719287 29772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:54:34.719332 29772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:34.719365 29772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:54:34.719384 29772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:34.727564 29772 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.2:44455
I20260809 08:54:34.727675 29959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.2:44455 every 8 connection(s)
I20260809 08:54:34.728142 29772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
I20260809 08:54:34.736025 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 29772
I20260809 08:54:34.736382 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.8.154.20:53291/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.3:34349
--local_ip_for_outbound_sockets=127.8.154.3
--tserver_master_addrs=127.8.154.62:36715
--webserver_port=37821
--webserver_interface=127.8.154.3
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:34033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.8.154.20:53291/kms with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 08:54:34 dist-test-slave-t3q3 krb5kdc[27871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265674, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:54:34.745237 29770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:34.733022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:48279 (local address 127.8.154.62:36715)
0809 08:54:34.733565 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:34.733568 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:34.733801 (+ 233us) server_negotiation.cc:407] Connection header received
0809 08:54:34.734633 (+ 832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:34.734634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:34.734651 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:34.734685 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:54:34.735615 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:34.736410 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:34.738171 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:34.738507 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:34.738556 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:34.741454 (+ 2898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:54:34.741456 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:54:34.741457 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:54:34.741466 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:54:34.743316 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:34.743893 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:34.743895 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:34.743895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:34.743964 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:34.744329 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:34.744330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:34.744331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:34.744493 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:54:34.744544 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:34.744980 (+ 436us) server_negotiation.cc:299] Negotiation successful
0809 08:54:34.745095 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508}
I20260809 08:54:34.745884 29968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:34.733179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:36715 (local address 127.8.154.2:48279)
0809 08:54:34.733652 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:34.733704 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 08:54:34.734486 (+ 782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:54:34.734905 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:54:34.734910 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:54:34.735190 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:54:34.735515 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:34.735524 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:34.736548 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:34.736549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:34.738013 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:34.738016 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:34.738676 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:34.738677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:34.738844 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:34.739482 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:54:34.739494 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:54:34.741332 (+ 1838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:54:34.743417 (+ 2085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:34.743425 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:34.743428 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:34.743781 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:34.744049 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:34.744049 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:34.744050 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:34.744232 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:34.744602 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:54:34.744608 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:54:34.744798 (+ 190us) client_negotiation.cc:769] Sending connection context
0809 08:54:34.745399 (+ 601us) client_negotiation.cc:240] Negotiation successful
0809 08:54:34.745589 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":95,"threads_started":1}
I20260809 08:54:34.746419 29960 heartbeater.cc:347] Connected to a master server at 127.8.154.62:36715
I20260809 08:54:34.746589 29960 heartbeater.cc:464] Registering TS with master...
I20260809 08:54:34.746867 29960 heartbeater.cc:511] Master 127.8.154.62:36715 requested a full tablet report, sending...
W20260809 08:54:34.747772 29960 heartbeater.cc:644] Failed to heartbeat to 127.8.154.62:36715 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 08:54:34.747911 29960 heartbeater.cc:464] Registering TS with master...
I20260809 08:54:34.748032 29960 heartbeater.cc:511] Master 127.8.154.62:36715 requested a full tablet report, sending...
I20260809 08:54:34.748514 29960 heartbeater.cc:464] Registering TS with master...
I20260809 08:54:34.748642 29960 heartbeater.cc:511] Master 127.8.154.62:36715 requested a full tablet report, sending...
W20260809 08:54:34.749013 29960 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 08:54:34.841049 29969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:54:34.841363 29969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:54:34.841401 29969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:54:34.843286 29969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:54:34.843350 29969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:54:34.843394 29969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.3
I20260809 08:54:34.846007 29969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:34033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.8.154.20:53291/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.3:34349
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.154.3
--webserver_port=37821
--tserver_master_addrs=127.8.154.62:36715
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.3
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:54:34.846292 29969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:54:34.846575 29969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:54:34.853745 29975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:34.853816 29974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:34.853732 29977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:54:34.854583 29969 server_base.cc:1034] running on GCE node
I20260809 08:54:34.854799 29969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:54:34.855072 29969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:54:34.856240 29969 hybrid_clock.cc:648] HybridClock initialized: now 1786265674856213 us; error 42 us; skew 500 ppm
Aug 09 08:54:34 dist-test-slave-t3q3 krb5kdc[27871](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265674, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:54:34.859032 29969 init.cc:377] Logged in from keytab as kudu/127.8.154.3@KRBTEST.COM (short username kudu)
I20260809 08:54:34.859591 29969 webserver.cc:460] Webserver started at http://127.8.154.3:37821/ using document root <none> and password file <none>
I20260809 08:54:34.859884 29969 fs_manager.cc:359] Metadata directory not provided
I20260809 08:54:34.859977 29969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 08:54:34 dist-test-slave-t3q3 krb5kdc[27871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265674, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for HTTP/127.8.154.20@KRBTEST.COM
I20260809 08:54:34.893321 29969 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.000s
I20260809 08:54:34.894618 29989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:34.894860 29969 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:54:34.895016 29969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-2/wal
uuid: "f2faa26ce0e94f2c999153e341b5f7fd"
format_stamp: "Formatted at 2026-08-09 08:54:33 on dist-test-slave-t3q3"
server_key: "800c132e7415f050d5c7e1ebcf25011b"
server_key_iv: "fc0b25a46ea8be1060190e45f704d39b"
server_key_version: "kuduclusterkey@0"
I20260809 08:54:34.895598 29969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:54:34.927003 29969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:54:34.929454 29969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:54:34.929653 29969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:54:34.929903 29969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:54:34.930377 29969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:54:34.930439 29969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:34.930485 29969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:54:34.930517 29969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:34.937602 29969 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.3:34349
I20260809 08:54:34.937685 30155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.3:34349 every 8 connection(s)
I20260809 08:54:34.938088 29969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
Aug 09 08:54:34 dist-test-slave-t3q3 krb5kdc[27871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265674, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:54:34.945544 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 29969
I20260809 08:54:34.949946 30161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:34.939021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:36715 (local address 127.8.154.3:53921)
0809 08:54:34.939298 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:34.939349 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 08:54:34.940002 (+ 653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:54:34.940321 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:54:34.940326 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:54:34.940551 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:54:34.940847 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:34.940855 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:34.941581 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:34.941582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:34.942612 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:34.942615 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:34.943065 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:34.943066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:34.943204 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:34.943789 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:54:34.943799 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:54:34.946071 (+ 2272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:54:34.948195 (+ 2124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:34.948204 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:34.948207 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:34.948516 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:34.948751 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:34.948752 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:34.948752 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:34.948906 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:34.949296 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:54:34.949305 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:54:34.949498 (+ 193us) client_negotiation.cc:769] Sending connection context
0809 08:54:34.949621 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 08:54:34.949767 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":119,"threads_started":1}
I20260809 08:54:34.950292 29770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:34.939027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:53921 (local address 127.8.154.62:36715)
0809 08:54:34.939104 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:34.939107 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:34.939454 (+ 347us) server_negotiation.cc:407] Connection header received
0809 08:54:34.940151 (+ 697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:34.940153 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:34.940171 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:34.940230 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:54:34.940939 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:34.941469 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:34.942708 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:34.942933 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:34.943027 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:34.946214 (+ 3187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:54:34.946216 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:54:34.946217 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:54:34.946225 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:54:34.948040 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:34.948618 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:34.948620 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:34.948620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:34.948693 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:34.948999 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:34.949000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:34.949000 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:34.949181 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:54:34.949261 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:34.950122 (+ 861us) server_negotiation.cc:299] Negotiation successful
0809 08:54:34.950219 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 08:54:34.950749 30156 heartbeater.cc:347] Connected to a master server at 127.8.154.62:36715
I20260809 08:54:34.950858 30156 heartbeater.cc:464] Registering TS with master...
I20260809 08:54:34.951057 30156 heartbeater.cc:511] Master 127.8.154.62:36715 requested a full tablet report, sending...
W20260809 08:54:34.951831 30156 heartbeater.cc:644] Failed to heartbeat to 127.8.154.62:36715 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 08:54:34.951926 30156 heartbeater.cc:464] Registering TS with master...
I20260809 08:54:34.952000 30156 heartbeater.cc:511] Master 127.8.154.62:36715 requested a full tablet report, sending...
I20260809 08:54:34.952423 30156 heartbeater.cc:464] Registering TS with master...
I20260809 08:54:34.952520 30156 heartbeater.cc:511] Master 127.8.154.62:36715 requested a full tablet report, sending...
W20260809 08:54:34.952877 30156 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 08:54:34.957276 30163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:34.946253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44374 (local address 127.8.154.62:36715)
0809 08:54:34.946584 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:34.946587 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:34.948606 (+ 2019us) server_negotiation.cc:407] Connection header received
0809 08:54:34.948628 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:34.948628 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:34.948643 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:34.950374 (+ 1731us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:54:34.950412 (+ 38us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:34.951059 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:34.952318 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:34.952670 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:34.952807 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:34.954330 (+ 1523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:54:34.954332 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:54:34.954332 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:54:34.954340 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:54:34.955926 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:34.956443 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:34.956445 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:34.956445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:34.956518 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:34.956740 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:34.956741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:34.956741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:34.956854 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:54:34.957032 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:34.957077 (+ 45us) server_negotiation.cc:299] Negotiation successful
0809 08:54:34.957158 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":71,"threads_started":1}
Aug 09 08:54:35 dist-test-slave-t3q3 krb5kdc[27871](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786265675, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:54:35.559794 29767 heartbeater.cc:464] Registering TS with master...
I20260809 08:54:35.560024 29767 heartbeater.cc:511] Master 127.8.154.62:36715 requested a full tablet report, sending...
I20260809 08:54:35.560685 29455 ts_manager.cc:194] Registered new tserver with Master: 4f1db1398004416390bec4da38bdc5a0 (127.8.154.1:34745)
I20260809 08:54:35.561722 29455 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.1@KRBTEST.COM'} at 127.8.154.1:52155
I20260809 08:54:35.749253 29960 heartbeater.cc:464] Registering TS with master...
I20260809 08:54:35.749433 29960 heartbeater.cc:511] Master 127.8.154.62:36715 requested a full tablet report, sending...
I20260809 08:54:35.749744 29455 ts_manager.cc:194] Registered new tserver with Master: 97f93d66119c41a8b37ff268557ac86f (127.8.154.2:44455)
I20260809 08:54:35.750304 29455 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.2@KRBTEST.COM'} at 127.8.154.2:48279
Aug 09 08:54:35 dist-test-slave-t3q3 krb5kdc[27871](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265675, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for HTTP/127.8.154.20@KRBTEST.COM
I20260809 08:54:35.953060 30156 heartbeater.cc:464] Registering TS with master...
I20260809 08:54:35.953236 30156 heartbeater.cc:511] Master 127.8.154.62:36715 requested a full tablet report, sending...
I20260809 08:54:35.953555 29455 ts_manager.cc:194] Registered new tserver with Master: f2faa26ce0e94f2c999153e341b5f7fd (127.8.154.3:34349)
I20260809 08:54:35.954103 29455 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.3@KRBTEST.COM'} at 127.8.154.3:53921
I20260809 08:54:35.961350 8808 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:54:35.972179 30189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:35.962618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44382 (local address 127.8.154.62:36715)
0809 08:54:35.962763 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:35.962765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:35.962923 (+ 158us) server_negotiation.cc:407] Connection header received
0809 08:54:35.963010 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:35.963011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:35.963024 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:35.963108 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:54:35.963924 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:35.964747 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:35.965991 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:35.966335 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:35.966408 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:35.968322 (+ 1914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:54:35.968324 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:54:35.968324 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:54:35.968332 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:54:35.970606 (+ 2274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:35.971126 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:35.971127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:35.971128 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:35.971196 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:35.971438 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:35.971439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:35.971439 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:35.971574 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:54:35.971677 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:35.971846 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 08:54:35.971935 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":88,"threads_started":1}
I20260809 08:54:35.974306 29455 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44382:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:54:35.976166 29455 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:54:35.987133 30198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:35.982031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59320 (local address 127.8.154.2:44455)
0809 08:54:35.982200 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:35.982204 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:35.982491 (+ 287us) server_negotiation.cc:407] Connection header received
0809 08:54:35.982537 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:35.982539 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:35.982557 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:35.982633 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:54:35.982811 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:35.983997 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:35.985642 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:35.986512 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:35.986628 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:35.986705 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:35.986983 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 08:54:35.987033 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":77,"threads_started":1}
I20260809 08:54:35.987138 30197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:35.981742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:44455 (local address 127.0.0.1:59320)
0809 08:54:35.982270 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:35.982313 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 08:54:35.982396 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:54:35.982654 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:54:35.982658 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:54:35.982660 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:54:35.982734 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:35.982736 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:35.984151 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:35.984152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:35.985537 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:35.985540 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:35.986751 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:35.986752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:35.986851 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:35.986867 (+ 16us) client_negotiation.cc:769] Sending connection context
0809 08:54:35.986959 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 08:54:35.987042 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":102,"threads_started":1}
I20260809 08:54:35.988492 29861 tablet_service.cc:1467] Processing CreateTablet for tablet 986fd81eeafe43338410b6df2d3a7adf (DEFAULT_TABLE table=test-table [id=22fa1b57ee05432e89f69bc33fe4aaff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:54:35.988952 29861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 986fd81eeafe43338410b6df2d3a7adf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:54:35.994201 30196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:35.981877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46422 (local address 127.8.154.3:34349)
0809 08:54:35.988184 (+ 6307us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:35.988189 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:35.988393 (+ 204us) server_negotiation.cc:407] Connection header received
0809 08:54:35.988439 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:35.988441 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:35.988459 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:35.988555 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:54:35.989134 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:35.990429 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:35.992914 (+ 2485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:35.993722 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:35.993823 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:35.993887 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:35.994063 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 08:54:35.994108 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6265,"thread_start_us":78,"threads_started":1}
I20260809 08:54:35.995103 30195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:35.981743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:34349 (local address 127.0.0.1:46422)
0809 08:54:35.988253 (+ 6510us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:35.988265 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:54:35.988330 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:54:35.988981 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:54:35.988982 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:54:35.988984 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:54:35.989034 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:35.989036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:35.990696 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:35.990696 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:35.992628 (+ 1932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:35.992632 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:35.993824 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:35.993825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:35.993897 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:35.993904 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 08:54:35.993974 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 08:54:35.995020 (+ 1046us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6491,"mutex_wait_us":1249,"thread_start_us":82,"threads_started":1}
I20260809 08:54:35.996181 30049 tablet_service.cc:1467] Processing CreateTablet for tablet 986fd81eeafe43338410b6df2d3a7adf (DEFAULT_TABLE table=test-table [id=22fa1b57ee05432e89f69bc33fe4aaff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:54:35.996595 30049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 986fd81eeafe43338410b6df2d3a7adf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:54:35.996773 30197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:35.981743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:34745 (local address 127.0.0.1:36820)
0809 08:54:35.987587 (+ 5844us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:35.987600 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:54:35.987650 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:54:35.990563 (+ 2913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:54:35.990564 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:54:35.990566 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:54:35.990612 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:35.990614 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:35.993924 (+ 3310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:35.993925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:35.994916 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:35.994920 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:35.996511 (+ 1591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:35.996512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:35.996583 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:35.996587 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:54:35.996649 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 08:54:35.996695 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5827,"mutex_wait_us":3,"thread_start_us":110,"threads_started":1}
I20260809 08:54:35.997129 30200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:35.982057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36820 (local address 127.8.154.1:34745)
0809 08:54:35.988715 (+ 6658us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:35.988720 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:35.988731 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:54:35.988770 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:35.988772 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:35.988788 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:35.988875 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:54:35.991592 (+ 2717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:35.992792 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:35.995445 (+ 2653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:35.996310 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:35.996396 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:35.996458 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:35.996984 (+ 526us) server_negotiation.cc:299] Negotiation successful
0809 08:54:35.997042 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6614,"thread_start_us":74,"threads_started":1}
I20260809 08:54:35.997519 30204 tablet_bootstrap.cc:492] T 986fd81eeafe43338410b6df2d3a7adf P 97f93d66119c41a8b37ff268557ac86f: Bootstrap starting.
I20260809 08:54:35.998270 30204 tablet_bootstrap.cc:654] T 986fd81eeafe43338410b6df2d3a7adf P 97f93d66119c41a8b37ff268557ac86f: Neither blocks nor log segments found. Creating new log.
I20260809 08:54:35.998759 30204 log.cc:824] T 986fd81eeafe43338410b6df2d3a7adf P 97f93d66119c41a8b37ff268557ac86f: Log is configured to *not* fsync() on all Append() calls
I20260809 08:54:35.999692 30204 tablet_bootstrap.cc:492] T 986fd81eeafe43338410b6df2d3a7adf P 97f93d66119c41a8b37ff268557ac86f: No bootstrap required, opened a new log
I20260809 08:54:35.999775 30204 ts_tablet_manager.cc:1397] T 986fd81eeafe43338410b6df2d3a7adf P 97f93d66119c41a8b37ff268557ac86f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:54:36.001238 29669 tablet_service.cc:1467] Processing CreateTablet for tablet 986fd81eeafe43338410b6df2d3a7adf (DEFAULT_TABLE table=test-table [id=22fa1b57ee05432e89f69bc33fe4aaff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:54:36.001623 29669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 986fd81eeafe43338410b6df2d3a7adf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:54:36.002307 30204 raft_consensus.cc:348] T 986fd81eeafe43338410b6df2d3a7adf P 97f93d66119c41a8b37ff268557ac86f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97f93d66119c41a8b37ff268557ac86f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 44455 } } peers { permanent_uuid: "f2faa26ce0e94f2c999153e341b5f7fd" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 34349 } } peers { permanent_uuid: "4f1db1398004416390bec4da38bdc5a0" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 34745 } }
I20260809 08:54:36.002511 30204 raft_consensus.cc:374] T 986fd81eeafe43338410b6df2d3a7adf P 97f93d66119c41a8b37ff268557ac86f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:54:36.002558 30204 raft_consensus.cc:729] T 986fd81eeafe43338410b6df2d3a7adf P 97f93d66119c41a8b37ff268557ac86f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97f93d66119c41a8b37ff268557ac86f, State: Initialized, Role: FOLLOWER
I20260809 08:54:36.002923 30204 consensus_queue.cc:260] T 986fd81eeafe43338410b6df2d3a7adf P 97f93d66119c41a8b37ff268557ac86f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97f93d66119c41a8b37ff268557ac86f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 44455 } } peers { permanent_uuid: "f2faa26ce0e94f2c999153e341b5f7fd" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 34349 } } peers { permanent_uuid: "4f1db1398004416390bec4da38bdc5a0" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 34745 } }
I20260809 08:54:36.003257 30204 ts_tablet_manager.cc:1428] T 986fd81eeafe43338410b6df2d3a7adf P 97f93d66119c41a8b37ff268557ac86f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:54:36.003403 29960 heartbeater.cc:503] Master 127.8.154.62:36715 was elected leader, sending a full tablet report...
I20260809 08:54:36.005549 30210 tablet_bootstrap.cc:492] T 986fd81eeafe43338410b6df2d3a7adf P 4f1db1398004416390bec4da38bdc5a0: Bootstrap starting.
I20260809 08:54:36.005837 30208 tablet_bootstrap.cc:492] T 986fd81eeafe43338410b6df2d3a7adf P f2faa26ce0e94f2c999153e341b5f7fd: Bootstrap starting.
I20260809 08:54:36.006270 30210 tablet_bootstrap.cc:654] T 986fd81eeafe43338410b6df2d3a7adf P 4f1db1398004416390bec4da38bdc5a0: Neither blocks nor log segments found. Creating new log.
I20260809 08:54:36.006525 30208 tablet_bootstrap.cc:654] T 986fd81eeafe43338410b6df2d3a7adf P f2faa26ce0e94f2c999153e341b5f7fd: Neither blocks nor log segments found. Creating new log.
I20260809 08:54:36.006659 30210 log.cc:824] T 986fd81eeafe43338410b6df2d3a7adf P 4f1db1398004416390bec4da38bdc5a0: Log is configured to *not* fsync() on all Append() calls
I20260809 08:54:36.006887 30208 log.cc:824] T 986fd81eeafe43338410b6df2d3a7adf P f2faa26ce0e94f2c999153e341b5f7fd: Log is configured to *not* fsync() on all Append() calls
I20260809 08:54:36.007546 30210 tablet_bootstrap.cc:492] T 986fd81eeafe43338410b6df2d3a7adf P 4f1db1398004416390bec4da38bdc5a0: No bootstrap required, opened a new log
I20260809 08:54:36.007627 30210 ts_tablet_manager.cc:1397] T 986fd81eeafe43338410b6df2d3a7adf P 4f1db1398004416390bec4da38bdc5a0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:54:36.008177 30208 tablet_bootstrap.cc:492] T 986fd81eeafe43338410b6df2d3a7adf P f2faa26ce0e94f2c999153e341b5f7fd: No bootstrap required, opened a new log
I20260809 08:54:36.008283 30208 ts_tablet_manager.cc:1397] T 986fd81eeafe43338410b6df2d3a7adf P f2faa26ce0e94f2c999153e341b5f7fd: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260809 08:54:36.010298 30210 raft_consensus.cc:348] T 986fd81eeafe43338410b6df2d3a7adf P 4f1db1398004416390bec4da38bdc5a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97f93d66119c41a8b37ff268557ac86f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 44455 } } peers { permanent_uuid: "f2faa26ce0e94f2c999153e341b5f7fd" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 34349 } } peers { permanent_uuid: "4f1db1398004416390bec4da38bdc5a0" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 34745 } }
I20260809 08:54:36.010504 30210 raft_consensus.cc:374] T 986fd81eeafe43338410b6df2d3a7adf P 4f1db1398004416390bec4da38bdc5a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:54:36.010555 30210 raft_consensus.cc:729] T 986fd81eeafe43338410b6df2d3a7adf P 4f1db1398004416390bec4da38bdc5a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f1db1398004416390bec4da38bdc5a0, State: Initialized, Role: FOLLOWER
I20260809 08:54:36.010979 30210 consensus_queue.cc:260] T 986fd81eeafe43338410b6df2d3a7adf P 4f1db1398004416390bec4da38bdc5a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97f93d66119c41a8b37ff268557ac86f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 44455 } } peers { permanent_uuid: "f2faa26ce0e94f2c999153e341b5f7fd" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 34349 } } peers { permanent_uuid: "4f1db1398004416390bec4da38bdc5a0" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 34745 } }
I20260809 08:54:36.011310 30210 ts_tablet_manager.cc:1428] T 986fd81eeafe43338410b6df2d3a7adf P 4f1db1398004416390bec4da38bdc5a0: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 08:54:36.011297 30208 raft_consensus.cc:348] T 986fd81eeafe43338410b6df2d3a7adf P f2faa26ce0e94f2c999153e341b5f7fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97f93d66119c41a8b37ff268557ac86f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 44455 } } peers { permanent_uuid: "f2faa26ce0e94f2c999153e341b5f7fd" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 34349 } } peers { permanent_uuid: "4f1db1398004416390bec4da38bdc5a0" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 34745 } }
I20260809 08:54:36.011466 29767 heartbeater.cc:503] Master 127.8.154.62:36715 was elected leader, sending a full tablet report...
I20260809 08:54:36.011484 30208 raft_consensus.cc:374] T 986fd81eeafe43338410b6df2d3a7adf P f2faa26ce0e94f2c999153e341b5f7fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:54:36.011545 30208 raft_consensus.cc:729] T 986fd81eeafe43338410b6df2d3a7adf P f2faa26ce0e94f2c999153e341b5f7fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2faa26ce0e94f2c999153e341b5f7fd, State: Initialized, Role: FOLLOWER
I20260809 08:54:36.011704 30208 consensus_queue.cc:260] T 986fd81eeafe43338410b6df2d3a7adf P f2faa26ce0e94f2c999153e341b5f7fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97f93d66119c41a8b37ff268557ac86f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 44455 } } peers { permanent_uuid: "f2faa26ce0e94f2c999153e341b5f7fd" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 34349 } } peers { permanent_uuid: "4f1db1398004416390bec4da38bdc5a0" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 34745 } }
I20260809 08:54:36.012286 30208 ts_tablet_manager.cc:1428] T 986fd81eeafe43338410b6df2d3a7adf P f2faa26ce0e94f2c999153e341b5f7fd: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260809 08:54:36.012434 30156 heartbeater.cc:503] Master 127.8.154.62:36715 was elected leader, sending a full tablet report...
I20260809 08:54:36.016114 30213 raft_consensus.cc:482] T 986fd81eeafe43338410b6df2d3a7adf P 4f1db1398004416390bec4da38bdc5a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:54:36.016234 30213 raft_consensus.cc:504] T 986fd81eeafe43338410b6df2d3a7adf P 4f1db1398004416390bec4da38bdc5a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97f93d66119c41a8b37ff268557ac86f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 44455 } } peers { permanent_uuid: "f2faa26ce0e94f2c999153e341b5f7fd" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 34349 } } peers { permanent_uuid: "4f1db1398004416390bec4da38bdc5a0" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 34745 } }
I20260809 08:54:36.016866 30213 leader_election.cc:288] T 986fd81eeafe43338410b6df2d3a7adf P 4f1db1398004416390bec4da38bdc5a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97f93d66119c41a8b37ff268557ac86f (127.8.154.2:44455), f2faa26ce0e94f2c999153e341b5f7fd (127.8.154.3:34349)
I20260809 08:54:36.021014 30198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:36.016842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:41987 (local address 127.8.154.2:44455)
0809 08:54:36.016928 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:36.016930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:36.017230 (+ 300us) server_negotiation.cc:407] Connection header received
0809 08:54:36.017330 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:36.017331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:36.017345 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:36.017375 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:54:36.017590 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:36.018330 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:36.019976 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:36.020587 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:36.020638 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:36.020647 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:36.020898 (+ 251us) server_negotiation.cc:299] Negotiation successful
0809 08:54:36.020948 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 08:54:36.021040 30196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:36.016836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:42029 (local address 127.8.154.3:34349)
0809 08:54:36.016890 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:36.016892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:36.017187 (+ 295us) server_negotiation.cc:407] Connection header received
0809 08:54:36.017301 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:36.017302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:36.017314 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:36.017346 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:54:36.017609 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:36.018328 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:36.019975 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:36.020594 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:36.020638 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:36.020647 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:36.020944 (+ 297us) server_negotiation.cc:299] Negotiation successful
0809 08:54:36.020982 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 08:54:36.021298 30216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:36.016703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:34349 (local address 127.8.154.1:42029)
0809 08:54:36.017111 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:36.017124 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:54:36.017218 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:54:36.017417 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:54:36.017418 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:54:36.017420 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:54:36.017507 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:36.017510 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:36.018621 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:36.018622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:36.019862 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:36.019865 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:36.020758 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:36.020758 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:36.020863 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:36.020866 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:54:36.021143 (+ 277us) client_negotiation.cc:240] Negotiation successful
0809 08:54:36.021199 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"mutex_wait_us":11,"thread_start_us":416,"threads_started":1}
I20260809 08:54:36.021610 30217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:36.016703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:44455 (local address 127.8.154.1:41987)
0809 08:54:36.017170 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:36.017180 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:54:36.017253 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:54:36.017437 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:54:36.017438 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:54:36.017439 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:54:36.017507 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:36.017509 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:36.018434 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:36.018435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:36.019862 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:36.019865 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:36.020731 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:36.020733 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:36.020809 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:36.020813 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:54:36.021499 (+ 686us) client_negotiation.cc:240] Negotiation successful
0809 08:54:36.021539 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"mutex_wait_us":17,"thread_start_us":353,"threads_started":1}
I20260809 08:54:36.021610 30088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986fd81eeafe43338410b6df2d3a7adf" candidate_uuid: "4f1db1398004416390bec4da38bdc5a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2faa26ce0e94f2c999153e341b5f7fd" is_pre_election: true
I20260809 08:54:36.021759 30088 raft_consensus.cc:2454] T 986fd81eeafe43338410b6df2d3a7adf P f2faa26ce0e94f2c999153e341b5f7fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1db1398004416390bec4da38bdc5a0 in term 0.
I20260809 08:54:36.021903 29895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986fd81eeafe43338410b6df2d3a7adf" candidate_uuid: "4f1db1398004416390bec4da38bdc5a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97f93d66119c41a8b37ff268557ac86f" is_pre_election: true
I20260809 08:54:36.022018 29589 leader_election.cc:302] T 986fd81eeafe43338410b6df2d3a7adf P 4f1db1398004416390bec4da38bdc5a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1db1398004416390bec4da38bdc5a0, f2faa26ce0e94f2c999153e341b5f7fd; no voters:
I20260809 08:54:36.022053 29895 raft_consensus.cc:2454] T 986fd81eeafe43338410b6df2d3a7adf P 97f93d66119c41a8b37ff268557ac86f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f1db1398004416390bec4da38bdc5a0 in term 0.
I20260809 08:54:36.022164 30213 raft_consensus.cc:2781] T 986fd81eeafe43338410b6df2d3a7adf P 4f1db1398004416390bec4da38bdc5a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:54:36.022220 30213 raft_consensus.cc:482] T 986fd81eeafe43338410b6df2d3a7adf P 4f1db1398004416390bec4da38bdc5a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:54:36.022245 30213 raft_consensus.cc:3037] T 986fd81eeafe43338410b6df2d3a7adf P 4f1db1398004416390bec4da38bdc5a0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:54:36.023170 30213 raft_consensus.cc:504] T 986fd81eeafe43338410b6df2d3a7adf P 4f1db1398004416390bec4da38bdc5a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97f93d66119c41a8b37ff268557ac86f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 44455 } } peers { permanent_uuid: "f2faa26ce0e94f2c999153e341b5f7fd" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 34349 } } peers { permanent_uuid: "4f1db1398004416390bec4da38bdc5a0" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 34745 } }
I20260809 08:54:36.023312 30213 leader_election.cc:288] T 986fd81eeafe43338410b6df2d3a7adf P 4f1db1398004416390bec4da38bdc5a0 [CANDIDATE]: Term 1 election: Requested vote from peers 97f93d66119c41a8b37ff268557ac86f (127.8.154.2:44455), f2faa26ce0e94f2c999153e341b5f7fd (127.8.154.3:34349)
I20260809 08:54:36.023523 29895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986fd81eeafe43338410b6df2d3a7adf" candidate_uuid: "4f1db1398004416390bec4da38bdc5a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97f93d66119c41a8b37ff268557ac86f"
I20260809 08:54:36.023577 30088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "986fd81eeafe43338410b6df2d3a7adf" candidate_uuid: "4f1db1398004416390bec4da38bdc5a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2faa26ce0e94f2c999153e341b5f7fd"
I20260809 08:54:36.023617 29895 raft_consensus.cc:3037] T 986fd81eeafe43338410b6df2d3a7adf P 97f93d66119c41a8b37ff268557ac86f [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:54:36.023643 30088 raft_consensus.cc:3037] T 986fd81eeafe43338410b6df2d3a7adf P f2faa26ce0e94f2c999153e341b5f7fd [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:54:36.024497 29895 raft_consensus.cc:2454] T 986fd81eeafe43338410b6df2d3a7adf P 97f93d66119c41a8b37ff268557ac86f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1db1398004416390bec4da38bdc5a0 in term 1.
I20260809 08:54:36.024497 30088 raft_consensus.cc:2454] T 986fd81eeafe43338410b6df2d3a7adf P f2faa26ce0e94f2c999153e341b5f7fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f1db1398004416390bec4da38bdc5a0 in term 1.
I20260809 08:54:36.024675 29589 leader_election.cc:302] T 986fd81eeafe43338410b6df2d3a7adf P 4f1db1398004416390bec4da38bdc5a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f1db1398004416390bec4da38bdc5a0, f2faa26ce0e94f2c999153e341b5f7fd; no voters:
I20260809 08:54:36.024757 30213 raft_consensus.cc:2781] T 986fd81eeafe43338410b6df2d3a7adf P 4f1db1398004416390bec4da38bdc5a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:54:36.024906 30213 raft_consensus.cc:686] T 986fd81eeafe43338410b6df2d3a7adf P 4f1db1398004416390bec4da38bdc5a0 [term 1 LEADER]: Becoming Leader. State: Replica: 4f1db1398004416390bec4da38bdc5a0, State: Running, Role: LEADER
I20260809 08:54:36.025002 30213 consensus_queue.cc:237] T 986fd81eeafe43338410b6df2d3a7adf P 4f1db1398004416390bec4da38bdc5a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97f93d66119c41a8b37ff268557ac86f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 44455 } } peers { permanent_uuid: "f2faa26ce0e94f2c999153e341b5f7fd" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 34349 } } peers { permanent_uuid: "4f1db1398004416390bec4da38bdc5a0" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 34745 } }
I20260809 08:54:36.025691 29454 catalog_manager.cc:5495] T 986fd81eeafe43338410b6df2d3a7adf P 4f1db1398004416390bec4da38bdc5a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f1db1398004416390bec4da38bdc5a0 (127.8.154.1). New cstate: current_term: 1 leader_uuid: "4f1db1398004416390bec4da38bdc5a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97f93d66119c41a8b37ff268557ac86f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 44455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2faa26ce0e94f2c999153e341b5f7fd" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 34349 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f1db1398004416390bec4da38bdc5a0" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 34745 } health_report { overall_health: HEALTHY } } }
I20260809 08:54:36.032487 30200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:36.030045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36830 (local address 127.8.154.1:34745)
0809 08:54:36.030093 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:36.030095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:36.030141 (+ 46us) server_negotiation.cc:407] Connection header received
0809 08:54:36.030218 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:36.030219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:36.030237 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:36.030302 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:54:36.030417 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:36.030868 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:36.031607 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:36.031877 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:36.031953 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:36.032162 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:54:36.032262 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:54:36.032347 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:36.032388 (+ 41us) server_negotiation.cc:299] Negotiation successful
0809 08:54:36.032407 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 08:54:36.034173 30088 raft_consensus.cc:1261] T 986fd81eeafe43338410b6df2d3a7adf P f2faa26ce0e94f2c999153e341b5f7fd [term 1 FOLLOWER]: Refusing update from remote peer 4f1db1398004416390bec4da38bdc5a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:54:36.034194 29895 raft_consensus.cc:1261] T 986fd81eeafe43338410b6df2d3a7adf P 97f93d66119c41a8b37ff268557ac86f [term 1 FOLLOWER]: Refusing update from remote peer 4f1db1398004416390bec4da38bdc5a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:54:36.034485 30213 consensus_queue.cc:1035] T 986fd81eeafe43338410b6df2d3a7adf P 4f1db1398004416390bec4da38bdc5a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2faa26ce0e94f2c999153e341b5f7fd" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 34349 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:54:36.034646 30213 consensus_queue.cc:1035] T 986fd81eeafe43338410b6df2d3a7adf P 4f1db1398004416390bec4da38bdc5a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "97f93d66119c41a8b37ff268557ac86f" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 44455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:54:36.037987 30227 mvcc.cc:205] Tried to move back new op lower bound from 7316544209033846784 to 7316544208998989824. Current Snapshot: MvccSnapshot[applied={T|T < 7316544209033846784}]
I20260809 08:54:36.039911 30229 mvcc.cc:205] Tried to move back new op lower bound from 7316544209033846784 to 7316544208998989824. Current Snapshot: MvccSnapshot[applied={T|T < 7316544209033846784}]
I20260809 08:54:36.041320 29454 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:54:36.041409 29454 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:54:36.042163 29454 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 827c858b4a3641c1b751f2981370df0f: Sending DeleteTablet for 3 replicas of tablet 986fd81eeafe43338410b6df2d3a7adf
I20260809 08:54:36.042508 30049 tablet_service.cc:1514] Processing DeleteTablet for tablet 986fd81eeafe43338410b6df2d3a7adf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:54:36 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:46422
I20260809 08:54:36.042551 29861 tablet_service.cc:1514] Processing DeleteTablet for tablet 986fd81eeafe43338410b6df2d3a7adf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:54:36 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:59320
I20260809 08:54:36.042913 29669 tablet_service.cc:1514] Processing DeleteTablet for tablet 986fd81eeafe43338410b6df2d3a7adf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:54:36 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:36820
I20260809 08:54:36.043166 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 29509
I20260809 08:54:36.045848 30241 tablet_replica.cc:330] T 986fd81eeafe43338410b6df2d3a7adf P 97f93d66119c41a8b37ff268557ac86f: stopping tablet replica
I20260809 08:54:36.045949 30241 raft_consensus.cc:2229] T 986fd81eeafe43338410b6df2d3a7adf P 97f93d66119c41a8b37ff268557ac86f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:54:36.046046 30241 raft_consensus.cc:2258] T 986fd81eeafe43338410b6df2d3a7adf P 97f93d66119c41a8b37ff268557ac86f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:54:36.046510 30241 ts_tablet_manager.cc:1905] T 986fd81eeafe43338410b6df2d3a7adf P 97f93d66119c41a8b37ff268557ac86f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:54:36.048660 30241 ts_tablet_manager.cc:1918] T 986fd81eeafe43338410b6df2d3a7adf P 97f93d66119c41a8b37ff268557ac86f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:54:36.048727 30241 log.cc:1196] T 986fd81eeafe43338410b6df2d3a7adf P 97f93d66119c41a8b37ff268557ac86f: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-1/wal/wals/986fd81eeafe43338410b6df2d3a7adf
I20260809 08:54:36.048988 30241 ts_tablet_manager.cc:1939] T 986fd81eeafe43338410b6df2d3a7adf P 97f93d66119c41a8b37ff268557ac86f: Deleting consensus metadata
I20260809 08:54:36.049384 29435 catalog_manager.cc:4841] TS 97f93d66119c41a8b37ff268557ac86f (127.8.154.2:44455): tablet 986fd81eeafe43338410b6df2d3a7adf (table test-table [id=22fa1b57ee05432e89f69bc33fe4aaff]) successfully deleted
I20260809 08:54:36.049576 30240 tablet_replica.cc:330] T 986fd81eeafe43338410b6df2d3a7adf P f2faa26ce0e94f2c999153e341b5f7fd: stopping tablet replica
I20260809 08:54:36.049698 30240 raft_consensus.cc:2229] T 986fd81eeafe43338410b6df2d3a7adf P f2faa26ce0e94f2c999153e341b5f7fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:54:36.049829 30240 raft_consensus.cc:2258] T 986fd81eeafe43338410b6df2d3a7adf P f2faa26ce0e94f2c999153e341b5f7fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:54:36.052968 30240 ts_tablet_manager.cc:1905] T 986fd81eeafe43338410b6df2d3a7adf P f2faa26ce0e94f2c999153e341b5f7fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:54:36.054888 30240 ts_tablet_manager.cc:1918] T 986fd81eeafe43338410b6df2d3a7adf P f2faa26ce0e94f2c999153e341b5f7fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:54:36.054973 30240 log.cc:1196] T 986fd81eeafe43338410b6df2d3a7adf P f2faa26ce0e94f2c999153e341b5f7fd: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265519942966-8808-0/minicluster-data/ts-2/wal/wals/986fd81eeafe43338410b6df2d3a7adf
I20260809 08:54:36.055243 30240 ts_tablet_manager.cc:1939] T 986fd81eeafe43338410b6df2d3a7adf P f2faa26ce0e94f2c999153e341b5f7fd: Deleting consensus metadata
I20260809 08:54:36.055665 29433 catalog_manager.cc:4841] TS f2faa26ce0e94f2c999153e341b5f7fd (127.8.154.3:34349): tablet 986fd81eeafe43338410b6df2d3a7adf (table test-table [id=22fa1b57ee05432e89f69bc33fe4aaff]) successfully deleted
W20260809 08:54:36.058252 29432 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.154.1:34745 (error 108)
I20260809 08:54:36.058647 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 29772
I20260809 08:54:36.068742 30197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:36.068591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:34745 (local address 127.0.0.1:36834)
0809 08:54:36.068650 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:36.068696 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.1:34745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 08:54:36.068918 29432 catalog_manager.cc:4584] TS 4f1db1398004416390bec4da38bdc5a0 (127.8.154.1:34745): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 986fd81eeafe43338410b6df2d3a7adf: Network error: Client connection negotiation failed: client connection to 127.8.154.1:34745: connect: Connection refused (error 111)
I20260809 08:54:36.070832 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 29969
I20260809 08:54:36.078490 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 29396
I20260809 08:54:36.094384 8808 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 08:54:36.452096 8808 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 08:54:36.452207 8808 mini_ranger.cc:66] Stopping Ranger...
I20260809 08:54:36.806684 8808 mini_ranger.cc:68] Stopped Ranger
2026-08-09 08:54:36.806 UTC [27921] LOG: received smart shutdown request
2026-08-09 08:54:36.811 UTC [27921] LOG: background worker "logical replication launcher" (PID 27935) exited with exit code 1
2026-08-09 08:54:36.811 UTC [27929] LOG: shutting down
2026-08-09 08:54:36.963 UTC [27921] LOG: database system is shut down
2026-08-09T08:54:36Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (48588 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:54:37 dist-test-slave-t3q3 krb5kdc[30257](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:54:37 dist-test-slave-t3q3 krb5kdc[30257](info): set up 2 sockets
Aug 09 08:54:37 dist-test-slave-t3q3 krb5kdc[30257](info): commencing operation
krb5kdc: starting...
W20260809 08:54:38.345321 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.237s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:54:38 dist-test-slave-t3q3 krb5kdc[30257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:54:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:54:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
I20260809 08:54:38.467262 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:46235
--webserver_interface=127.8.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:35479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:46235,127.8.154.61:35989,127.8.154.60:45617
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:54:38.549945 30276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:54:38.550153 30276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:54:38.550175 30276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:54:38.551218 30276 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:54:38.551252 30276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:54:38.551266 30276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:54:38.551280 30276 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:54:38.551290 30276 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:54:38.552911 30276 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:35479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:46235,127.8.154.61:35989,127.8.154.60:45617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:46235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30276
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:54:38.553182 30276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:54:38.553362 30276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:54:38.558220 30282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:38.558264 30285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:38.558220 30287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:54:38.558559 30276 server_base.cc:1034] running on GCE node
I20260809 08:54:38.558830 30276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:54:38.559120 30276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:54:38.560285 30276 hybrid_clock.cc:648] HybridClock initialized: now 1786265678560282 us; error 39 us; skew 500 ppm
Aug 09 08:54:38 dist-test-slave-t3q3 krb5kdc[30257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265678, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:54:38.566039 30276 init.cc:377] Logged in from keytab as kudu/127.8.154.62@KRBTEST.COM (short username kudu)
I20260809 08:54:38.566489 30276 webserver.cc:460] Webserver started at http://127.8.154.62:44891/ using document root <none> and password file <none>
I20260809 08:54:38.566725 30276 fs_manager.cc:359] Metadata directory not provided
I20260809 08:54:38.566795 30276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:54:38.566955 30276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:54:38.568033 30276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "838304c2512147f19490ca4bcc90ad59"
format_stamp: "Formatted at 2026-08-09 08:54:38 on dist-test-slave-t3q3"
server_key: "07cc608f2ff034299133cc40853d5897"
server_key_iv: "2393f0abfbefcf3aff3dc336f8e01514"
server_key_version: "encryptionkey@0"
I20260809 08:54:38.568470 30276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "838304c2512147f19490ca4bcc90ad59"
format_stamp: "Formatted at 2026-08-09 08:54:38 on dist-test-slave-t3q3"
server_key: "07cc608f2ff034299133cc40853d5897"
server_key_iv: "2393f0abfbefcf3aff3dc336f8e01514"
server_key_version: "encryptionkey@0"
I20260809 08:54:38.569805 30276 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:54:38.570554 30299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:38.570719 30276 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:54:38.570824 30276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "838304c2512147f19490ca4bcc90ad59"
format_stamp: "Formatted at 2026-08-09 08:54:38 on dist-test-slave-t3q3"
server_key: "07cc608f2ff034299133cc40853d5897"
server_key_iv: "2393f0abfbefcf3aff3dc336f8e01514"
server_key_version: "encryptionkey@0"
I20260809 08:54:38.570905 30276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:54:38.588033 30276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:54:38.590446 30276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:54:38.590613 30276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:54:38.594695 30276 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:46235
I20260809 08:54:38.594733 30373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:46235 every 8 connection(s)
I20260809 08:54:38.595122 30276 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:54:38.596231 30374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:54:38.597836 30374 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46235 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:54:38.603248 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 30276
I20260809 08:54:38.603380 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:54:38.603600 8808 external_mini_cluster.cc:1428] Setting key 2de64aa505da1e03bb19e66aaf1772bd
Aug 09 08:54:38 dist-test-slave-t3q3 krb5kdc[30257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265678, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:54:38.607721 30380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:38.598722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41094 (local address 127.8.154.62:46235)
0809 08:54:38.598844 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:38.598852 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:38.598880 (+ 28us) server_negotiation.cc:407] Connection header received
0809 08:54:38.599372 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:38.599373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:38.599390 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:38.599468 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:54:38.599982 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.600848 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.601721 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.602028 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.602111 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:38.604456 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:54:38.604465 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:54:38.604465 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:54:38.604474 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:54:38.606088 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:38.606565 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:38.606568 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:38.606568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:38.606662 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:38.606935 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:38.606935 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:38.606936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:38.607146 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:54:38.607212 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:38.607451 (+ 239us) server_negotiation.cc:299] Negotiation successful
0809 08:54:38.607546 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":62,"threads_started":1}
I20260809 08:54:38.607849 30379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:38.598562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46235 (local address 127.0.0.1:41094)
0809 08:54:38.598730 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:38.598769 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 08:54:38.599240 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:54:38.599496 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:54:38.599498 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:54:38.599745 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:54:38.599902 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:38.599906 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.600940 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.600941 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:38.601647 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:38.601649 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.602109 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.602110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:38.602172 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:38.602632 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:54:38.602641 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:54:38.604364 (+ 1723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:54:38.606176 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:38.606183 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:38.606186 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:38.606466 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:38.606732 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:38.606753 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:38.606753 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:38.606866 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:38.607218 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:54:38.607226 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:54:38.607305 (+ 79us) client_negotiation.cc:769] Sending connection context
0809 08:54:38.607408 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 08:54:38.607658 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":79,"thread_start_us":74,"threads_started":1}
I20260809 08:54:38.608644 30374 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35989 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:54:38.609040 30379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:38.608899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.61:35989 (local address 127.0.0.1:33338)
0809 08:54:38.608953 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:38.609007 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.61:35989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 08:54:38.609193 30303 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.154.61:35989: connect: Connection refused (error 111)
I20260809 08:54:38.609666 30379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:38.609550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.61:35989 (local address 127.0.0.1:33350)
0809 08:54:38.609600 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:38.609634 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.61:35989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 08:54:38.609826 30374 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.154.61:35989: Network error: Client connection negotiation failed: client connection to 127.8.154.61:35989: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.8.154.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.61.keytab.
Entry for principal kudu/127.8.154.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.61.keytab.
I20260809 08:54:38.628937 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.61:35989
--webserver_interface=127.8.154.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.61.keytab
--principal=kudu/127.8.154.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:35479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:46235,127.8.154.61:35989,127.8.154.60:45617
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:54:38.659085 30374 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35989 } attempt: 1
I20260809 08:54:38.659641 30379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:38.659474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.61:35989 (local address 127.0.0.1:33352)
0809 08:54:38.659545 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:38.659604 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.61:35989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 08:54:38.660017 30379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:38.659923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.61:35989 (local address 127.0.0.1:33368)
0809 08:54:38.659967 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:38.659996 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.61:35989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 08:54:38.660149 30374 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.154.61:35989: Network error: Client connection negotiation failed: client connection to 127.8.154.61:35989: connect: Connection refused (error 111)
W20260809 08:54:38.712469 30385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:54:38.712672 30385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:54:38.712694 30385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:54:38.713840 30385 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:54:38.713884 30385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:54:38.713899 30385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:54:38.713913 30385 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:54:38.713925 30385 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:54:38.715409 30385 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:35479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:46235,127.8.154.61:35989,127.8.154.60:45617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.61.keytab
--principal=kudu/127.8.154.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.61:35989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.154.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:54:38.715657 30385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:54:38.715880 30385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:54:38.720831 30397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:38.720804 30392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:38.720825 30395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:54:38.721217 30385 server_base.cc:1034] running on GCE node
I20260809 08:54:38.721480 30385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:54:38.721781 30385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:54:38.722947 30385 hybrid_clock.cc:648] HybridClock initialized: now 1786265678722936 us; error 32 us; skew 500 ppm
Aug 09 08:54:38 dist-test-slave-t3q3 krb5kdc[30257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265678, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:54:38.725672 30385 init.cc:377] Logged in from keytab as kudu/127.8.154.61@KRBTEST.COM (short username kudu)
I20260809 08:54:38.726156 30385 webserver.cc:460] Webserver started at http://127.8.154.61:33483/ using document root <none> and password file <none>
I20260809 08:54:38.726389 30385 fs_manager.cc:359] Metadata directory not provided
I20260809 08:54:38.726461 30385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:54:38.726608 30385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:54:38.727690 30385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-1/data/instance:
uuid: "f75f913fde814d899503bcdafb807987"
format_stamp: "Formatted at 2026-08-09 08:54:38 on dist-test-slave-t3q3"
server_key: "87c229134817547a8140039a70c9f8ff"
server_key_iv: "710e0d922a19544df9545ccf6bd506b4"
server_key_version: "encryptionkey@0"
I20260809 08:54:38.728128 30385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-1/wal/instance:
uuid: "f75f913fde814d899503bcdafb807987"
format_stamp: "Formatted at 2026-08-09 08:54:38 on dist-test-slave-t3q3"
server_key: "87c229134817547a8140039a70c9f8ff"
server_key_iv: "710e0d922a19544df9545ccf6bd506b4"
server_key_version: "encryptionkey@0"
I20260809 08:54:38.728324 30374 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35989 } attempt: 2
I20260809 08:54:38.728819 30379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:38.728631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.61:35989 (local address 127.0.0.1:33376)
0809 08:54:38.728698 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:38.728732 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.61:35989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 08:54:38.729254 30379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:38.729140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.61:35989 (local address 127.0.0.1:33386)
0809 08:54:38.729181 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:38.729204 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.61:35989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 08:54:38.729351 30374 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.154.61:35989: Network error: Client connection negotiation failed: client connection to 127.8.154.61:35989: connect: Connection refused (error 111)
I20260809 08:54:38.730110 30385 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260809 08:54:38.731273 30411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:38.731581 30385 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:54:38.731690 30385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-1/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-1/wal
uuid: "f75f913fde814d899503bcdafb807987"
format_stamp: "Formatted at 2026-08-09 08:54:38 on dist-test-slave-t3q3"
server_key: "87c229134817547a8140039a70c9f8ff"
server_key_iv: "710e0d922a19544df9545ccf6bd506b4"
server_key_version: "encryptionkey@0"
I20260809 08:54:38.731775 30385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:54:38.742077 30385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:54:38.744465 30385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:54:38.744637 30385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:54:38.748840 30385 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.61:35989
I20260809 08:54:38.748894 30488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.61:35989 every 8 connection(s)
I20260809 08:54:38.749248 30385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-1/data/info.pb
I20260809 08:54:38.750111 30489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:54:38.751688 30489 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46235 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:54:38.754422 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 30385
I20260809 08:54:38.754557 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-1/wal/instance
I20260809 08:54:38.754792 8808 external_mini_cluster.cc:1428] Setting key ade80339623d7e50ab6a29b05ae3d2d5
Aug 09 08:54:38 dist-test-slave-t3q3 krb5kdc[30257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265678, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.61@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:54:38.761924 30380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:38.752460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41098 (local address 127.8.154.62:46235)
0809 08:54:38.752526 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:38.752528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:38.752730 (+ 202us) server_negotiation.cc:407] Connection header received
0809 08:54:38.753201 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:38.753202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:38.753215 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:38.753275 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:54:38.753858 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.754335 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.755794 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.755990 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.756042 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:38.758688 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:54:38.758690 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:54:38.758690 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:54:38.758697 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:54:38.760325 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:38.760875 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:38.760876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:38.760876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:38.760936 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:38.761251 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:38.761251 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:38.761252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:38.761365 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:54:38.761419 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:38.761783 (+ 364us) server_negotiation.cc:299] Negotiation successful
0809 08:54:38.761852 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 08:54:38.762207 30493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:38.752425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46235 (local address 127.0.0.1:41098)
0809 08:54:38.752599 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:38.752641 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 08:54:38.753117 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:54:38.753348 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:54:38.753351 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:54:38.753587 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:54:38.753784 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:38.753788 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.754428 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.754430 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:38.755694 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:38.755698 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.756108 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.756109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:38.756268 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:38.756837 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:54:38.756846 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:54:38.758573 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:54:38.760414 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:38.760422 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:38.760425 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:38.760776 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:38.761008 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:38.761009 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:38.761010 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:38.761174 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:38.761439 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:54:38.761446 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:54:38.761631 (+ 185us) client_negotiation.cc:769] Sending connection context
0809 08:54:38.761757 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 08:54:38.761931 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":99,"threads_started":1}
I20260809 08:54:38.762867 30489 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35989 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.154.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.60@KRBTEST.COM" created.
Aug 09 08:54:38 dist-test-slave-t3q3 krb5kdc[30257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265678, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.61@KRBTEST.COM for kudu/127.8.154.61@KRBTEST.COM
I20260809 08:54:38.771448 30495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:38.763239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33396 (local address 127.8.154.61:35989)
0809 08:54:38.763397 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:38.763401 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:38.763417 (+ 16us) server_negotiation.cc:407] Connection header received
0809 08:54:38.763433 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:38.763435 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:38.763446 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:38.763529 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:54:38.763808 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.764611 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.765305 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.765612 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.765693 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:38.768234 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:54:38.768244 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:54:38.768244 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:54:38.768252 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:54:38.770058 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:38.770520 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:38.770524 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:38.770524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:38.770581 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:38.770810 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:38.770810 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:38.770811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:38.770961 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:54:38.771027 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:38.771209 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 08:54:38.771359 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":133,"thread_start_us":83,"threads_started":1}
I20260809 08:54:38.771497 30493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:38.763167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.61:35989 (local address 127.0.0.1:33396)
0809 08:54:38.763236 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:38.763244 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 08:54:38.763278 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:54:38.763541 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:54:38.763542 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:54:38.763681 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:54:38.763729 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:38.763730 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.764707 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.764708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:38.765223 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:38.765225 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.765689 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.765690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:38.765762 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:38.766148 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:54:38.766153 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:54:38.768139 (+ 1986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:54:38.770171 (+ 2032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:38.770173 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:38.770173 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:38.770421 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:38.770658 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:38.770659 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:38.770659 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:38.770733 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:38.771033 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:54:38.771034 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:54:38.771090 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 08:54:38.771170 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 08:54:38.771295 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 08:54:38.772226 30489 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.154.60" port: 45617 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:54:38.772650 30493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:38.772500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:45617 (local address 127.0.0.1:60028)
0809 08:54:38.772552 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:38.772616 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.60:45617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 08:54:38.772779 30412 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.154.60:45617: connect: Connection refused (error 111)
I20260809 08:54:38.773221 30493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:38.773129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:45617 (local address 127.0.0.1:60040)
0809 08:54:38.773176 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:38.773202 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.60:45617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 08:54:38.773353 30489 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.154.60:45617: Network error: Client connection negotiation failed: client connection to 127.8.154.60:45617: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.60.keytab.
Entry for principal kudu/127.8.154.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.60.keytab.
I20260809 08:54:38.784667 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.60:45617
--webserver_interface=127.8.154.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.60.keytab
--principal=kudu/127.8.154.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:35479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:46235,127.8.154.61:35989,127.8.154.60:45617
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:54:38.820510 30374 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35989 } attempt: 3
I20260809 08:54:38.822614 30489 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.154.60" port: 45617 } attempt: 1
I20260809 08:54:38.823079 30493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:38.822908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:45617 (local address 127.0.0.1:60048)
0809 08:54:38.822975 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:38.823028 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.60:45617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 08:54:38.823518 30493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:38.823370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:45617 (local address 127.0.0.1:60060)
0809 08:54:38.823431 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:38.823471 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.60:45617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 08:54:38.823623 30489 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.154.60:45617: Network error: Client connection negotiation failed: client connection to 127.8.154.60:45617: connect: Connection refused (error 111)
Aug 09 08:54:38 dist-test-slave-t3q3 krb5kdc[30257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265678, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for kudu/127.8.154.61@KRBTEST.COM
I20260809 08:54:38.828447 30495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:38.821071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33412 (local address 127.8.154.61:35989)
0809 08:54:38.821147 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:38.821149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:38.821161 (+ 12us) server_negotiation.cc:407] Connection header received
0809 08:54:38.821223 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:38.821224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:38.821238 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:38.821316 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:54:38.821666 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.822174 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.823094 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.823579 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.823947 (+ 368us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:38.825888 (+ 1941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:54:38.825889 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:54:38.825889 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:54:38.825896 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:54:38.827220 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:38.827657 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:38.827659 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:38.827659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:38.827718 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:38.827947 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:38.827948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:38.827948 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:38.828055 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:54:38.828173 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:38.828328 (+ 155us) server_negotiation.cc:299] Negotiation successful
0809 08:54:38.828393 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 08:54:38.828516 30379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:38.820985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.61:35989 (local address 127.0.0.1:33412)
0809 08:54:38.821070 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:38.821081 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:54:38.821158 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:54:38.821326 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:54:38.821328 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:54:38.821504 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:54:38.821576 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:38.821578 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.822275 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.822276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:38.822990 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:38.822993 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.823681 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.823682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:38.823772 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:38.824319 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:54:38.824324 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:54:38.825801 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:54:38.827305 (+ 1504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:38.827307 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:38.827307 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:38.827545 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:38.827806 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:38.827807 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:38.827807 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:38.827882 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:38.828169 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:54:38.828171 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:54:38.828224 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 08:54:38.828310 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 08:54:38.828412 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 08:54:38.829018 30374 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.154.60" port: 45617 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:54:38.829382 30379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:38.829268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:45617 (local address 127.0.0.1:60062)
0809 08:54:38.829320 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:38.829356 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.60:45617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 08:54:38.829715 30379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:38.829626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:45617 (local address 127.0.0.1:60064)
0809 08:54:38.829670 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:38.829697 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.60:45617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 08:54:38.829811 30374 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.154.60:45617: Network error: Client connection negotiation failed: client connection to 127.8.154.60:45617: connect: Connection refused (error 111)
I20260809 08:54:38.860966 30374 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.154.60" port: 45617 } attempt: 1
I20260809 08:54:38.861491 30379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:38.861321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:45617 (local address 127.0.0.1:60076)
0809 08:54:38.861394 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:38.861445 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.60:45617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 08:54:38.861866 30379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:38.861773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:45617 (local address 127.0.0.1:60090)
0809 08:54:38.861804 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:38.861838 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.60:45617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260809 08:54:38.862048 30374 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.154.60:45617: Network error: Client connection negotiation failed: client connection to 127.8.154.60:45617: connect: Connection refused (error 111)
W20260809 08:54:38.872475 30500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:54:38.872702 30500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:54:38.872740 30500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:54:38.873961 30500 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:54:38.874035 30500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:54:38.874063 30500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:54:38.874106 30500 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:54:38.874136 30500 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:54:38.875667 30500 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:35479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:46235,127.8.154.61:35989,127.8.154.60:45617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.60.keytab
--principal=kudu/127.8.154.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.60:45617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.154.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:54:38.875962 30500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:54:38.876230 30500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:54:38.881220 30507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:38.881238 30508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:38.881201 30513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:54:38.881388 30500 server_base.cc:1034] running on GCE node
I20260809 08:54:38.881814 30500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:54:38.882085 30500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:54:38.883239 30500 hybrid_clock.cc:648] HybridClock initialized: now 1786265678883226 us; error 32 us; skew 500 ppm
Aug 09 08:54:38 dist-test-slave-t3q3 krb5kdc[30257](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265678, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:54:38.885789 30500 init.cc:377] Logged in from keytab as kudu/127.8.154.60@KRBTEST.COM (short username kudu)
I20260809 08:54:38.886204 30500 webserver.cc:460] Webserver started at http://127.8.154.60:42779/ using document root <none> and password file <none>
I20260809 08:54:38.886442 30500 fs_manager.cc:359] Metadata directory not provided
I20260809 08:54:38.886509 30500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:54:38.886662 30500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:54:38.887699 30500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-2/data/instance:
uuid: "2cb34214d910400581456f25722697c1"
format_stamp: "Formatted at 2026-08-09 08:54:38 on dist-test-slave-t3q3"
server_key: "17c6423c00baa273133055850e2b8668"
server_key_iv: "185b2b624248949105a86c28fcedd50f"
server_key_version: "encryptionkey@0"
I20260809 08:54:38.888123 30500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-2/wal/instance:
uuid: "2cb34214d910400581456f25722697c1"
format_stamp: "Formatted at 2026-08-09 08:54:38 on dist-test-slave-t3q3"
server_key: "17c6423c00baa273133055850e2b8668"
server_key_iv: "185b2b624248949105a86c28fcedd50f"
server_key_version: "encryptionkey@0"
I20260809 08:54:38.889463 30500 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:54:38.890278 30521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:38.890475 30500 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:54:38.890565 30500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-2/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-2/wal
uuid: "2cb34214d910400581456f25722697c1"
format_stamp: "Formatted at 2026-08-09 08:54:38 on dist-test-slave-t3q3"
server_key: "17c6423c00baa273133055850e2b8668"
server_key_iv: "185b2b624248949105a86c28fcedd50f"
server_key_version: "encryptionkey@0"
I20260809 08:54:38.890658 30500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:54:38.891768 30489 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.154.60" port: 45617 } attempt: 2
I20260809 08:54:38.892181 30493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:38.892035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:45617 (local address 127.0.0.1:60106)
0809 08:54:38.892117 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:38.892153 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.60:45617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 08:54:38.892554 30493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:38.892459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:45617 (local address 127.0.0.1:60116)
0809 08:54:38.892489 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:38.892531 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.60:45617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260809 08:54:38.892663 30489 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.154.60:45617: Network error: Client connection negotiation failed: client connection to 127.8.154.60:45617: connect: Connection refused (error 111)
I20260809 08:54:38.910033 30500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:54:38.911916 30500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:54:38.912147 30500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:54:38.915999 30500 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.60:45617
I20260809 08:54:38.916127 30604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.60:45617 every 8 connection(s)
I20260809 08:54:38.916437 30500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-2/data/info.pb
I20260809 08:54:38.917313 30605 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:54:38.918912 30605 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46235 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:54:38.920668 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 30500
I20260809 08:54:38.920763 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265519942966-8808-0/minicluster-data/master-2/wal/instance
I20260809 08:54:38.920976 8808 external_mini_cluster.cc:1428] Setting key 3dec68162a908859391a7faf2401ac42
Aug 09 08:54:38 dist-test-slave-t3q3 krb5kdc[30257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265678, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.60@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
Aug 09 08:54:38 dist-test-slave-t3q3 krb5kdc[30257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.154.60@KRBTEST.COM
I20260809 08:54:38.929508 30380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:38.919675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41100 (local address 127.8.154.62:46235)
0809 08:54:38.919772 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:38.919775 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:38.919939 (+ 164us) server_negotiation.cc:407] Connection header received
0809 08:54:38.920537 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:38.920538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:38.920551 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:38.920614 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:54:38.921468 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.921986 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.923167 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.923443 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.923499 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:38.926326 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:54:38.926327 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:54:38.926328 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:54:38.926336 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:54:38.928108 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:38.928521 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:38.928523 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:38.928523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:38.928598 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:38.928855 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:38.928856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:38.928857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:38.928981 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:54:38.929063 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:38.929328 (+ 265us) server_negotiation.cc:299] Negotiation successful
0809 08:54:38.929416 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 08:54:38.929715 30607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:38.919609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46235 (local address 127.0.0.1:41100)
0809 08:54:38.919806 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:38.919852 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 08:54:38.920401 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:54:38.920689 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:54:38.920693 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:54:38.921048 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:54:38.921356 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:38.921364 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.922086 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.922087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:38.923077 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:38.923081 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.923550 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.923551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:38.923703 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:38.924356 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:54:38.924366 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:54:38.926197 (+ 1831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:54:38.928188 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:38.928194 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:38.928197 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:38.928419 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:38.928668 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:38.928669 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:38.928669 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:38.928778 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:38.929048 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:54:38.929054 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:54:38.929194 (+ 140us) client_negotiation.cc:769] Sending connection context
0809 08:54:38.929306 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 08:54:38.929453 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":89,"threads_started":1}
I20260809 08:54:38.930301 30609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:38.921390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60132 (local address 127.8.154.60:45617)
0809 08:54:38.921561 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:38.921566 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:38.921617 (+ 51us) server_negotiation.cc:407] Connection header received
0809 08:54:38.921687 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:38.921689 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:38.921710 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:38.921777 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:54:38.922520 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.923285 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.924339 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.924629 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.924686 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:38.927473 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:54:38.927484 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:54:38.927484 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:54:38.927492 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:54:38.929025 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:38.929393 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:38.929396 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:38.929396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:38.929458 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:38.929683 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:38.929683 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:38.929684 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:38.929858 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:54:38.930045 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:38.930102 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 08:54:38.930206 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":107,"threads_started":1}
I20260809 08:54:38.930388 30605 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35989 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 08:54:38 dist-test-slave-t3q3 krb5kdc[30257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265678, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.60@KRBTEST.COM for kudu/127.8.154.61@KRBTEST.COM
Aug 09 08:54:38 dist-test-slave-t3q3 krb5kdc[30257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:54:38.937155 30607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:38.930671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.61:35989 (local address 127.0.0.1:33426)
0809 08:54:38.930715 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:38.930725 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:54:38.930762 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:54:38.930885 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:54:38.930885 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:54:38.931058 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:54:38.931124 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:38.931126 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.931747 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.931748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:38.932306 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:38.932308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.932680 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.932681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:38.932752 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:38.933163 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:54:38.933168 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:54:38.934437 (+ 1269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:54:38.936093 (+ 1656us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:38.936094 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:38.936095 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:38.936267 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:38.936500 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:38.936501 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:38.936501 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:38.936561 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:38.936831 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:54:38.936832 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:54:38.936890 (+ 58us) client_negotiation.cc:769] Sending connection context
0809 08:54:38.936969 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 08:54:38.937061 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 08:54:38.937162 30495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:38.930732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33426 (local address 127.8.154.61:35989)
0809 08:54:38.930778 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:38.930780 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:38.930788 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:54:38.930817 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:38.930818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:38.930827 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:38.930872 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:54:38.931206 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.931667 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.932387 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.932598 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.932671 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:38.934536 (+ 1865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:54:38.934537 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:54:38.934537 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:54:38.934544 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:54:38.936000 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:38.936363 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:38.936364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:38.936364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:38.936425 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:38.936628 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:38.936629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:38.936629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:38.936746 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:54:38.936824 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:38.936996 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 08:54:38.937069 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 08:54:38.937238 30374 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.154.60" port: 45617 } attempt: 2
I20260809 08:54:38.937892 30605 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.154.60" port: 45617 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:54:38.938875 30380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:38.931027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41106 (local address 127.8.154.62:46235)
0809 08:54:38.931073 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:38.931075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:38.931082 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:54:38.931114 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:38.931115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:38.931123 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:38.931167 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:54:38.931631 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.932121 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.932761 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.933078 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.933146 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:38.935689 (+ 2543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:54:38.935690 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:54:38.935691 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:54:38.935697 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:54:38.937291 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:38.937691 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:38.937691 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:38.937692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:38.937753 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:38.938280 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:38.938282 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:38.938282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:38.938431 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:54:38.938654 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:38.938691 (+ 37us) server_negotiation.cc:299] Negotiation successful
0809 08:54:38.938764 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Aug 09 08:54:38 dist-test-slave-t3q3 krb5kdc[30257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265678, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for kudu/127.8.154.60@KRBTEST.COM
Aug 09 08:54:38 dist-test-slave-t3q3 krb5kdc[30257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265678, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.60@KRBTEST.COM for kudu/127.8.154.60@KRBTEST.COM
Aug 09 08:54:38 dist-test-slave-t3q3 krb5kdc[30257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.154.61@KRBTEST.COM
I20260809 08:54:38.945680 30609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:38.937507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60146 (local address 127.8.154.60:45617)
0809 08:54:38.937711 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:38.937713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:38.937720 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:54:38.937739 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:38.937740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:38.937753 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:38.937830 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:54:38.938115 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.938812 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.939770 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.940074 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.940393 (+ 319us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:38.942108 (+ 1715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:54:38.942109 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:54:38.942109 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:54:38.942117 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:54:38.944493 (+ 2376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:38.944865 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:38.944866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:38.944867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:38.944925 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:38.945164 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:38.945165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:38.945165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:38.945290 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:54:38.945387 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:38.945564 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 08:54:38.945624 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184}
I20260809 08:54:38.945734 30379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:38.937464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:45617 (local address 127.0.0.1:60146)
0809 08:54:38.937509 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:38.937519 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:54:38.937568 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:54:38.937850 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:54:38.937851 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:54:38.938007 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:54:38.938049 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:38.938050 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.938908 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.938909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:38.939674 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:38.939676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.940198 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.940198 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:38.940251 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:38.940654 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:54:38.940659 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:54:38.942020 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:54:38.944616 (+ 2596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:38.944617 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:38.944617 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:38.944790 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:38.945021 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:38.945022 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:38.945022 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:38.945084 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:38.945388 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:54:38.945389 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:54:38.945441 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 08:54:38.945524 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 08:54:38.945612 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 08:54:38.947798 30607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:38.938155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:45617 (local address 127.0.0.1:60156)
0809 08:54:38.938356 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:38.938365 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:54:38.938405 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:54:38.938561 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:54:38.938561 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:54:38.938702 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:54:38.938726 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:38.938727 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.940163 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.940163 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:38.940884 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:38.940887 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.941364 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.941366 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:38.941433 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:38.941853 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:54:38.941858 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:54:38.943174 (+ 1316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:54:38.946593 (+ 3419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:38.946594 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:38.946595 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:38.946820 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:38.947039 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:38.947040 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:38.947040 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:38.947110 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:38.947441 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:54:38.947442 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:54:38.947490 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 08:54:38.947574 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 08:54:38.947674 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187}
I20260809 08:54:38.947861 30374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 838304c2512147f19490ca4bcc90ad59: Bootstrap starting.
I20260809 08:54:38.948195 30610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:38.938210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60156 (local address 127.8.154.60:45617)
0809 08:54:38.938383 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:38.938385 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:38.938394 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:54:38.938463 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:38.938464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:38.938473 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:38.938532 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:54:38.938800 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.940047 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.940971 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.941279 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.941355 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:38.944168 (+ 2813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:54:38.944169 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:54:38.944170 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:54:38.944176 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:54:38.946519 (+ 2343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:38.946899 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:38.946901 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:38.946901 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:38.946962 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:38.947184 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:38.947185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:38.947185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:38.947357 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:54:38.947433 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:38.947613 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 08:54:38.947742 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":150,"thread_start_us":118,"threads_started":1}
I20260809 08:54:38.948715 30374 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 838304c2512147f19490ca4bcc90ad59: Neither blocks nor log segments found. Creating new log.
I20260809 08:54:38.948823 30495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:38.939879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33434 (local address 127.8.154.61:35989)
0809 08:54:38.939915 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:38.939917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:38.939958 (+ 41us) server_negotiation.cc:407] Connection header received
0809 08:54:38.940027 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:38.940028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:38.940036 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:38.940154 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:54:38.940693 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.941298 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.942268 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.942609 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.942687 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:38.945445 (+ 2758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:54:38.945446 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:54:38.945447 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:54:38.945453 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:54:38.947432 (+ 1979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:38.947783 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:38.947784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:38.947785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:38.947840 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:38.948102 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:38.948103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:38.948104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:38.948244 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:54:38.948453 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:38.948658 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 08:54:38.948724 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 08:54:38.949076 30374 log.cc:824] T 00000000000000000000000000000000 P 838304c2512147f19490ca4bcc90ad59: Log is configured to *not* fsync() on all Append() calls
I20260809 08:54:38.950604 30374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 838304c2512147f19490ca4bcc90ad59: No bootstrap required, opened a new log
I20260809 08:54:38.950858 30605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cb34214d910400581456f25722697c1: Bootstrap starting.
I20260809 08:54:38.951336 8808 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:54:38.951599 30605 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2cb34214d910400581456f25722697c1: Neither blocks nor log segments found. Creating new log.
I20260809 08:54:38.951879 30605 log.cc:824] T 00000000000000000000000000000000 P 2cb34214d910400581456f25722697c1: Log is configured to *not* fsync() on all Append() calls
I20260809 08:54:38.952687 30605 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cb34214d910400581456f25722697c1: No bootstrap required, opened a new log
I20260809 08:54:38.952679 30374 raft_consensus.cc:348] T 00000000000000000000000000000000 P 838304c2512147f19490ca4bcc90ad59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838304c2512147f19490ca4bcc90ad59" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46235 } } peers { permanent_uuid: "f75f913fde814d899503bcdafb807987" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35989 } } peers { permanent_uuid: "2cb34214d910400581456f25722697c1" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 45617 } }
I20260809 08:54:38.952829 30374 raft_consensus.cc:374] T 00000000000000000000000000000000 P 838304c2512147f19490ca4bcc90ad59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:54:38.952875 30374 raft_consensus.cc:729] T 00000000000000000000000000000000 P 838304c2512147f19490ca4bcc90ad59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 838304c2512147f19490ca4bcc90ad59, State: Initialized, Role: FOLLOWER
I20260809 08:54:38.953024 30374 consensus_queue.cc:260] T 00000000000000000000000000000000 P 838304c2512147f19490ca4bcc90ad59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838304c2512147f19490ca4bcc90ad59" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46235 } } peers { permanent_uuid: "f75f913fde814d899503bcdafb807987" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35989 } } peers { permanent_uuid: "2cb34214d910400581456f25722697c1" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 45617 } }
I20260809 08:54:38.953378 30374 sys_catalog.cc:564] T 00000000000000000000000000000000 P 838304c2512147f19490ca4bcc90ad59 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:54:38.954207 30626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 838304c2512147f19490ca4bcc90ad59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838304c2512147f19490ca4bcc90ad59" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46235 } } peers { permanent_uuid: "f75f913fde814d899503bcdafb807987" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35989 } } peers { permanent_uuid: "2cb34214d910400581456f25722697c1" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 45617 } } }
I20260809 08:54:38.954368 30626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 838304c2512147f19490ca4bcc90ad59 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:54:38.955003 30605 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2cb34214d910400581456f25722697c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838304c2512147f19490ca4bcc90ad59" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46235 } } peers { permanent_uuid: "f75f913fde814d899503bcdafb807987" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35989 } } peers { permanent_uuid: "2cb34214d910400581456f25722697c1" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 45617 } }
I20260809 08:54:38.955140 30605 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2cb34214d910400581456f25722697c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:54:38.955171 30605 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2cb34214d910400581456f25722697c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cb34214d910400581456f25722697c1, State: Initialized, Role: FOLLOWER
I20260809 08:54:38.955273 30605 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2cb34214d910400581456f25722697c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838304c2512147f19490ca4bcc90ad59" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46235 } } peers { permanent_uuid: "f75f913fde814d899503bcdafb807987" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35989 } } peers { permanent_uuid: "2cb34214d910400581456f25722697c1" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 45617 } }
I20260809 08:54:38.955554 30605 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2cb34214d910400581456f25722697c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:54:38.956910 30276 master_runner.cc:186] Error getting master registration for 127.8.154.61:35989: OK, instance_id { permanent_uuid: "f75f913fde814d899503bcdafb807987" instance_seqno: 1786265678747614 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 08:54:38.960534 30276 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 08:54:38.957335 30642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cb34214d910400581456f25722697c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838304c2512147f19490ca4bcc90ad59" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46235 } } peers { permanent_uuid: "f75f913fde814d899503bcdafb807987" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35989 } } peers { permanent_uuid: "2cb34214d910400581456f25722697c1" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 45617 } } }
I20260809 08:54:38.960644 30642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cb34214d910400581456f25722697c1 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 08:54:38.958006 30666 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2cb34214d910400581456f25722697c1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:54:38.960711 30666 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260809 08:54:38.962018 30641 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 838304c2512147f19490ca4bcc90ad59: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:54:38.962066 30641 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:54:38.962352 30380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:38.953363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41110 (local address 127.8.154.62:46235)
0809 08:54:38.953487 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:38.953489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:38.953499 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:54:38.953519 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:38.953520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:38.953530 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:38.953591 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:54:38.954867 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.955525 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.956601 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.956880 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.956917 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:38.958958 (+ 2041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:54:38.958960 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:54:38.958960 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:54:38.958967 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:54:38.960839 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:38.961214 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:38.961215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:38.961216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:38.961274 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:38.961468 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:38.961469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:38.961469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:38.961625 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:54:38.961737 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:38.962185 (+ 448us) server_negotiation.cc:299] Negotiation successful
0809 08:54:38.962261 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 08:54:38.962155 30500 master_runner.cc:186] Error getting master registration for 127.8.154.61:35989: OK, instance_id { permanent_uuid: "f75f913fde814d899503bcdafb807987" instance_seqno: 1786265678747614 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 08:54:38.964145 30500 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 08:54:38.967222 30495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:38.953043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33450 (local address 127.8.154.61:35989)
0809 08:54:38.953080 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:38.953081 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:38.953720 (+ 639us) server_negotiation.cc:407] Connection header received
0809 08:54:38.953933 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:38.953934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:38.953944 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:38.953972 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:54:38.954671 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.955359 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.956476 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.956754 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.956793 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:38.958409 (+ 1616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:54:38.958411 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:54:38.958411 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:54:38.958418 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:54:38.960198 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:38.964584 (+ 4386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:38.964585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:38.964585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:38.964651 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:38.965177 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:38.965178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:38.965179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:38.965310 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:54:38.965402 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:38.967010 (+ 1608us) server_negotiation.cc:299] Negotiation successful
0809 08:54:38.967129 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 08:54:38.969033 30610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:38.953461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60160 (local address 127.8.154.60:45617)
0809 08:54:38.957014 (+ 3553us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:38.957016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:38.957024 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:54:38.957043 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:38.957044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:38.957053 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:38.957107 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:54:38.960829 (+ 3722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.961600 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.962784 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.963101 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.963167 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:38.964720 (+ 1553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:54:38.964721 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:54:38.964721 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:54:38.964728 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:54:38.966613 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:38.967509 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:38.967511 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:38.967511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:38.967570 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:38.967786 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:38.967787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:38.967787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:38.967911 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:54:38.968472 (+ 561us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:38.968861 (+ 389us) server_negotiation.cc:299] Negotiation successful
0809 08:54:38.968945 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3533}
I20260809 08:54:38.983788 30489 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.154.60" port: 45617 } attempt: 3
Aug 09 08:54:38 dist-test-slave-t3q3 krb5kdc[30257](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265678, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.61@KRBTEST.COM for kudu/127.8.154.60@KRBTEST.COM
I20260809 08:54:38.992300 30493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:38.984075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:45617 (local address 127.0.0.1:60170)
0809 08:54:38.984152 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:38.984163 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:54:38.984204 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:54:38.984337 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:54:38.984338 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:54:38.984513 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:54:38.984565 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:38.984567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.985481 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.985482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:38.986306 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:38.986308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.986786 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.986787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:38.986851 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:38.987345 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:54:38.987350 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:54:38.989303 (+ 1953us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:54:38.991152 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:38.991153 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:38.991153 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:38.991370 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:38.991603 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:38.991604 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:38.991605 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:38.991672 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:38.991949 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:54:38.991950 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:54:38.991997 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 08:54:38.992099 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 08:54:38.992197 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 08:54:38.992314 30610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:38.984167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60170 (local address 127.8.154.60:45617)
0809 08:54:38.984216 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:38.984218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:38.984227 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:54:38.984250 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:38.984251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:38.984260 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:38.984313 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:54:38.984653 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.985397 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.986397 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:38.986705 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:38.986777 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:38.989394 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:54:38.989395 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:54:38.989395 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:54:38.989402 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:54:38.991065 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:38.991458 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:38.991460 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:38.991461 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:38.991528 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:38.991749 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:38.991750 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:38.991750 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:38.991879 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:54:38.991947 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:38.992144 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 08:54:38.992224 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 08:54:38.994246 30489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f75f913fde814d899503bcdafb807987: Bootstrap starting.
I20260809 08:54:38.995085 30489 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f75f913fde814d899503bcdafb807987: Neither blocks nor log segments found. Creating new log.
I20260809 08:54:38.995406 30489 log.cc:824] T 00000000000000000000000000000000 P f75f913fde814d899503bcdafb807987: Log is configured to *not* fsync() on all Append() calls
I20260809 08:54:38.996369 30489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f75f913fde814d899503bcdafb807987: No bootstrap required, opened a new log
I20260809 08:54:38.998412 30489 raft_consensus.cc:348] T 00000000000000000000000000000000 P f75f913fde814d899503bcdafb807987 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838304c2512147f19490ca4bcc90ad59" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46235 } } peers { permanent_uuid: "f75f913fde814d899503bcdafb807987" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35989 } } peers { permanent_uuid: "2cb34214d910400581456f25722697c1" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 45617 } }
I20260809 08:54:38.998553 30489 raft_consensus.cc:374] T 00000000000000000000000000000000 P f75f913fde814d899503bcdafb807987 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:54:38.998589 30489 raft_consensus.cc:729] T 00000000000000000000000000000000 P f75f913fde814d899503bcdafb807987 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f75f913fde814d899503bcdafb807987, State: Initialized, Role: FOLLOWER
I20260809 08:54:38.998710 30489 consensus_queue.cc:260] T 00000000000000000000000000000000 P f75f913fde814d899503bcdafb807987 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838304c2512147f19490ca4bcc90ad59" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46235 } } peers { permanent_uuid: "f75f913fde814d899503bcdafb807987" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35989 } } peers { permanent_uuid: "2cb34214d910400581456f25722697c1" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 45617 } }
I20260809 08:54:38.999023 30489 sys_catalog.cc:564] T 00000000000000000000000000000000 P f75f913fde814d899503bcdafb807987 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:54:38.998980 30670 sys_catalog.cc:455] T 00000000000000000000000000000000 P f75f913fde814d899503bcdafb807987 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838304c2512147f19490ca4bcc90ad59" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46235 } } peers { permanent_uuid: "f75f913fde814d899503bcdafb807987" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35989 } } peers { permanent_uuid: "2cb34214d910400581456f25722697c1" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 45617 } } }
I20260809 08:54:39.000633 30670 sys_catalog.cc:458] T 00000000000000000000000000000000 P f75f913fde814d899503bcdafb807987 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 08:54:39.001106 30691 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f75f913fde814d899503bcdafb807987: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:54:39.001188 30691 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:54:39.002852 30385 master_runner.cc:263] No leader master found from master f75f913fde814d899503bcdafb807987
I20260809 08:54:39.002899 30385 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 08:54:39.091408 30307 tablet.cc:2369] T 00000000000000000000000000000000 P 838304c2512147f19490ca4bcc90ad59: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:54:39.126072 30626 raft_consensus.cc:482] T 00000000000000000000000000000000 P 838304c2512147f19490ca4bcc90ad59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:54:39.126232 30626 raft_consensus.cc:504] T 00000000000000000000000000000000 P 838304c2512147f19490ca4bcc90ad59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838304c2512147f19490ca4bcc90ad59" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46235 } } peers { permanent_uuid: "f75f913fde814d899503bcdafb807987" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35989 } } peers { permanent_uuid: "2cb34214d910400581456f25722697c1" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 45617 } }
I20260809 08:54:39.126493 30626 leader_election.cc:288] T 00000000000000000000000000000000 P 838304c2512147f19490ca4bcc90ad59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f75f913fde814d899503bcdafb807987 (127.8.154.61:35989), 2cb34214d910400581456f25722697c1 (127.8.154.60:45617)
I20260809 08:54:39.126843 30449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "838304c2512147f19490ca4bcc90ad59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f75f913fde814d899503bcdafb807987" is_pre_election: true
I20260809 08:54:39.127008 30449 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f75f913fde814d899503bcdafb807987 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 838304c2512147f19490ca4bcc90ad59 in term 0.
I20260809 08:54:39.126940 30569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "838304c2512147f19490ca4bcc90ad59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cb34214d910400581456f25722697c1" is_pre_election: true
I20260809 08:54:39.127097 30569 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2cb34214d910400581456f25722697c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 838304c2512147f19490ca4bcc90ad59 in term 0.
I20260809 08:54:39.127306 30300 leader_election.cc:302] T 00000000000000000000000000000000 P 838304c2512147f19490ca4bcc90ad59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cb34214d910400581456f25722697c1, 838304c2512147f19490ca4bcc90ad59; no voters:
I20260809 08:54:39.127480 30626 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 838304c2512147f19490ca4bcc90ad59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:54:39.127542 30626 raft_consensus.cc:482] T 00000000000000000000000000000000 P 838304c2512147f19490ca4bcc90ad59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:54:39.127568 30626 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 838304c2512147f19490ca4bcc90ad59 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:54:39.128628 30626 raft_consensus.cc:504] T 00000000000000000000000000000000 P 838304c2512147f19490ca4bcc90ad59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838304c2512147f19490ca4bcc90ad59" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46235 } } peers { permanent_uuid: "f75f913fde814d899503bcdafb807987" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35989 } } peers { permanent_uuid: "2cb34214d910400581456f25722697c1" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 45617 } }
I20260809 08:54:39.128769 30626 leader_election.cc:288] T 00000000000000000000000000000000 P 838304c2512147f19490ca4bcc90ad59 [CANDIDATE]: Term 1 election: Requested vote from peers f75f913fde814d899503bcdafb807987 (127.8.154.61:35989), 2cb34214d910400581456f25722697c1 (127.8.154.60:45617)
I20260809 08:54:39.128964 30569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "838304c2512147f19490ca4bcc90ad59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cb34214d910400581456f25722697c1"
I20260809 08:54:39.129016 30449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "838304c2512147f19490ca4bcc90ad59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f75f913fde814d899503bcdafb807987"
I20260809 08:54:39.129050 30569 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2cb34214d910400581456f25722697c1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:54:39.129108 30449 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f75f913fde814d899503bcdafb807987 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:54:39.129977 30569 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2cb34214d910400581456f25722697c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 838304c2512147f19490ca4bcc90ad59 in term 1.
I20260809 08:54:39.130112 30449 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f75f913fde814d899503bcdafb807987 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 838304c2512147f19490ca4bcc90ad59 in term 1.
I20260809 08:54:39.130146 30300 leader_election.cc:302] T 00000000000000000000000000000000 P 838304c2512147f19490ca4bcc90ad59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cb34214d910400581456f25722697c1, 838304c2512147f19490ca4bcc90ad59; no voters:
I20260809 08:54:39.130223 30626 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 838304c2512147f19490ca4bcc90ad59 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:54:39.130352 30626 raft_consensus.cc:686] T 00000000000000000000000000000000 P 838304c2512147f19490ca4bcc90ad59 [term 1 LEADER]: Becoming Leader. State: Replica: 838304c2512147f19490ca4bcc90ad59, State: Running, Role: LEADER
I20260809 08:54:39.130471 30626 consensus_queue.cc:237] T 00000000000000000000000000000000 P 838304c2512147f19490ca4bcc90ad59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838304c2512147f19490ca4bcc90ad59" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46235 } } peers { permanent_uuid: "f75f913fde814d899503bcdafb807987" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35989 } } peers { permanent_uuid: "2cb34214d910400581456f25722697c1" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 45617 } }
I20260809 08:54:39.130934 30694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 838304c2512147f19490ca4bcc90ad59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 838304c2512147f19490ca4bcc90ad59. Latest consensus state: current_term: 1 leader_uuid: "838304c2512147f19490ca4bcc90ad59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838304c2512147f19490ca4bcc90ad59" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46235 } } peers { permanent_uuid: "f75f913fde814d899503bcdafb807987" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35989 } } peers { permanent_uuid: "2cb34214d910400581456f25722697c1" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 45617 } } }
I20260809 08:54:39.131026 30694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 838304c2512147f19490ca4bcc90ad59 [sys.catalog]: This master's current role is: LEADER
I20260809 08:54:39.131237 30696 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:54:39.131438 30696 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:54:39.133080 30449 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f75f913fde814d899503bcdafb807987 [term 1 FOLLOWER]: Refusing update from remote peer 838304c2512147f19490ca4bcc90ad59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:54:39.133093 30569 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2cb34214d910400581456f25722697c1 [term 1 FOLLOWER]: Refusing update from remote peer 838304c2512147f19490ca4bcc90ad59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:54:39.133394 30694 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 838304c2512147f19490ca4bcc90ad59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cb34214d910400581456f25722697c1" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 45617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:54:39.133488 30626 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 838304c2512147f19490ca4bcc90ad59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f75f913fde814d899503bcdafb807987" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:54:39.134742 30642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cb34214d910400581456f25722697c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 838304c2512147f19490ca4bcc90ad59. Latest consensus state: current_term: 1 leader_uuid: "838304c2512147f19490ca4bcc90ad59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838304c2512147f19490ca4bcc90ad59" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46235 } } peers { permanent_uuid: "f75f913fde814d899503bcdafb807987" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35989 } } peers { permanent_uuid: "2cb34214d910400581456f25722697c1" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 45617 } } }
I20260809 08:54:39.134747 30670 sys_catalog.cc:455] T 00000000000000000000000000000000 P f75f913fde814d899503bcdafb807987 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 838304c2512147f19490ca4bcc90ad59. Latest consensus state: current_term: 1 leader_uuid: "838304c2512147f19490ca4bcc90ad59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838304c2512147f19490ca4bcc90ad59" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46235 } } peers { permanent_uuid: "f75f913fde814d899503bcdafb807987" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35989 } } peers { permanent_uuid: "2cb34214d910400581456f25722697c1" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 45617 } } }
I20260809 08:54:39.134829 30670 sys_catalog.cc:458] T 00000000000000000000000000000000 P f75f913fde814d899503bcdafb807987 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:54:39.134829 30642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cb34214d910400581456f25722697c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:54:39.135732 30694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 838304c2512147f19490ca4bcc90ad59 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "838304c2512147f19490ca4bcc90ad59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838304c2512147f19490ca4bcc90ad59" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46235 } } peers { permanent_uuid: "f75f913fde814d899503bcdafb807987" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35989 } } peers { permanent_uuid: "2cb34214d910400581456f25722697c1" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 45617 } } }
I20260809 08:54:39.135825 30694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 838304c2512147f19490ca4bcc90ad59 [sys.catalog]: This master's current role is: LEADER
I20260809 08:54:39.136399 30670 sys_catalog.cc:455] T 00000000000000000000000000000000 P f75f913fde814d899503bcdafb807987 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "838304c2512147f19490ca4bcc90ad59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838304c2512147f19490ca4bcc90ad59" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46235 } } peers { permanent_uuid: "f75f913fde814d899503bcdafb807987" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35989 } } peers { permanent_uuid: "2cb34214d910400581456f25722697c1" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 45617 } } }
I20260809 08:54:39.136473 30670 sys_catalog.cc:458] T 00000000000000000000000000000000 P f75f913fde814d899503bcdafb807987 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:54:39.136634 30626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 838304c2512147f19490ca4bcc90ad59 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "838304c2512147f19490ca4bcc90ad59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838304c2512147f19490ca4bcc90ad59" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46235 } } peers { permanent_uuid: "f75f913fde814d899503bcdafb807987" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35989 } } peers { permanent_uuid: "2cb34214d910400581456f25722697c1" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 45617 } } }
I20260809 08:54:39.136722 30626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 838304c2512147f19490ca4bcc90ad59 [sys.catalog]: This master's current role is: LEADER
I20260809 08:54:39.136929 30699 mvcc.cc:205] Tried to move back new op lower bound from 7316544221726674944 to 7316544221719121920. Current Snapshot: MvccSnapshot[applied={T|T < 7316544221726674944}]
I20260809 08:54:39.137012 30697 mvcc.cc:205] Tried to move back new op lower bound from 7316544221726674944 to 7316544221719121920. Current Snapshot: MvccSnapshot[applied={T|T < 7316544221726674944}]
I20260809 08:54:39.137151 30642 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cb34214d910400581456f25722697c1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "838304c2512147f19490ca4bcc90ad59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "838304c2512147f19490ca4bcc90ad59" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46235 } } peers { permanent_uuid: "f75f913fde814d899503bcdafb807987" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35989 } } peers { permanent_uuid: "2cb34214d910400581456f25722697c1" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 45617 } } }
I20260809 08:54:39.137231 30642 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cb34214d910400581456f25722697c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:54:39.137401 30696 catalog_manager.cc:1347] Generated new cluster ID: 012b216bf69643e6bda85938bcc4ebed
I20260809 08:54:39.137437 30696 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:54:39.138293 30698 mvcc.cc:205] Tried to move back new op lower bound from 7316544221726674944 to 7316544221719121920. Current Snapshot: MvccSnapshot[applied={T|T < 7316544221726674944}]
I20260809 08:54:39.162573 30696 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:54:39.163285 30696 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:54:39.174165 30696 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 838304c2512147f19490ca4bcc90ad59: Generated new TSK 0
I20260809 08:54:39.174620 30696 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:54:39.961160 30666 catalog_manager.cc:1259] Loaded cluster ID: 012b216bf69643e6bda85938bcc4ebed
I20260809 08:54:39.961234 30666 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2cb34214d910400581456f25722697c1: loading cluster ID for follower catalog manager: success
I20260809 08:54:39.962082 30666 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2cb34214d910400581456f25722697c1: acquiring CA information for follower catalog manager: success
I20260809 08:54:39.962417 30666 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2cb34214d910400581456f25722697c1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 08:54:40.001637 30691 catalog_manager.cc:1259] Loaded cluster ID: 012b216bf69643e6bda85938bcc4ebed
I20260809 08:54:40.001717 30691 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f75f913fde814d899503bcdafb807987: loading cluster ID for follower catalog manager: success
I20260809 08:54:40.002475 30691 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f75f913fde814d899503bcdafb807987: acquiring CA information for follower catalog manager: success
I20260809 08:54:40.002732 30691 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f75f913fde814d899503bcdafb807987: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 08:54:40.226802 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 30276
I20260809 08:54:40.239813 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 30385
I20260809 08:54:40.250124 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 30500
2026-08-09T08:54:40Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3167 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:54:40 dist-test-slave-t3q3 krb5kdc[30715](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:54:40 dist-test-slave-t3q3 krb5kdc[30715](info): set up 2 sockets
Aug 09 08:54:40 dist-test-slave-t3q3 krb5kdc[30715](info): commencing operation
krb5kdc: starting...
W20260809 08:54:41.532569 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.257s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:54:41 dist-test-slave-t3q3 krb5kdc[30715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:54:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:54:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
I20260809 08:54:41.658432 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:39957
--webserver_interface=127.8.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:43415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:39957
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:54:41.739665 30742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:54:41.739900 30742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:54:41.739940 30742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:54:41.741336 30742 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:54:41.741391 30742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:54:41.741417 30742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:54:41.741451 30742 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:54:41.741467 30742 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:54:41.743535 30742 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:43415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:39957
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:39957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:54:41.743810 30742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:54:41.744029 30742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:54:41.748975 30748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:41.749025 30749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:41.748963 30751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:54:41.749297 30742 server_base.cc:1034] running on GCE node
I20260809 08:54:41.749565 30742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:54:41.749891 30742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:54:41.751060 30742 hybrid_clock.cc:648] HybridClock initialized: now 1786265681751046 us; error 27 us; skew 500 ppm
Aug 09 08:54:41 dist-test-slave-t3q3 krb5kdc[30715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265681, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:54:41.753599 30742 init.cc:377] Logged in from keytab as kudu/127.8.154.62@KRBTEST.COM (short username kudu)
I20260809 08:54:41.753973 30742 webserver.cc:460] Webserver started at http://127.8.154.62:43291/ using document root <none> and password file <none>
I20260809 08:54:41.754164 30742 fs_manager.cc:359] Metadata directory not provided
I20260809 08:54:41.754205 30742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:54:41.754294 30742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:54:41.755275 30742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "ff5039613fa14a0fa43b9dcffbff6972"
format_stamp: "Formatted at 2026-08-09 08:54:41 on dist-test-slave-t3q3"
server_key: "a566d59b7c0a9648f6112f37e6fb6c81"
server_key_iv: "a8ed8566b78684a4fdee466d4598281c"
server_key_version: "encryptionkey@0"
I20260809 08:54:41.755575 30742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "ff5039613fa14a0fa43b9dcffbff6972"
format_stamp: "Formatted at 2026-08-09 08:54:41 on dist-test-slave-t3q3"
server_key: "a566d59b7c0a9648f6112f37e6fb6c81"
server_key_iv: "a8ed8566b78684a4fdee466d4598281c"
server_key_version: "encryptionkey@0"
I20260809 08:54:41.756933 30742 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:54:41.757617 30759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:41.757804 30742 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:54:41.757879 30742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "ff5039613fa14a0fa43b9dcffbff6972"
format_stamp: "Formatted at 2026-08-09 08:54:41 on dist-test-slave-t3q3"
server_key: "a566d59b7c0a9648f6112f37e6fb6c81"
server_key_iv: "a8ed8566b78684a4fdee466d4598281c"
server_key_version: "encryptionkey@0"
I20260809 08:54:41.757977 30742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:54:41.768013 30742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:54:41.769829 30742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:54:41.769984 30742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:54:41.774091 30742 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:39957
I20260809 08:54:41.774139 30834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:39957 every 8 connection(s)
I20260809 08:54:41.774487 30742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:54:41.775355 30836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:54:41.777673 30836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff5039613fa14a0fa43b9dcffbff6972: Bootstrap starting.
I20260809 08:54:41.778259 30836 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff5039613fa14a0fa43b9dcffbff6972: Neither blocks nor log segments found. Creating new log.
I20260809 08:54:41.778550 30836 log.cc:824] T 00000000000000000000000000000000 P ff5039613fa14a0fa43b9dcffbff6972: Log is configured to *not* fsync() on all Append() calls
I20260809 08:54:41.779263 30836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff5039613fa14a0fa43b9dcffbff6972: No bootstrap required, opened a new log
I20260809 08:54:41.780895 30836 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff5039613fa14a0fa43b9dcffbff6972 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff5039613fa14a0fa43b9dcffbff6972" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 39957 } }
I20260809 08:54:41.781055 30836 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff5039613fa14a0fa43b9dcffbff6972 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:54:41.781101 30836 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff5039613fa14a0fa43b9dcffbff6972 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff5039613fa14a0fa43b9dcffbff6972, State: Initialized, Role: FOLLOWER
I20260809 08:54:41.781280 30836 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff5039613fa14a0fa43b9dcffbff6972 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff5039613fa14a0fa43b9dcffbff6972" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 39957 } }
I20260809 08:54:41.781380 30836 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff5039613fa14a0fa43b9dcffbff6972 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:54:41.781424 30836 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff5039613fa14a0fa43b9dcffbff6972 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:54:41.781476 30836 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff5039613fa14a0fa43b9dcffbff6972 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:54:41.781982 30836 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff5039613fa14a0fa43b9dcffbff6972 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff5039613fa14a0fa43b9dcffbff6972" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 39957 } }
I20260809 08:54:41.782101 30836 leader_election.cc:302] T 00000000000000000000000000000000 P ff5039613fa14a0fa43b9dcffbff6972 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff5039613fa14a0fa43b9dcffbff6972; no voters:
I20260809 08:54:41.782285 30836 leader_election.cc:288] T 00000000000000000000000000000000 P ff5039613fa14a0fa43b9dcffbff6972 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:54:41.782335 30840 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff5039613fa14a0fa43b9dcffbff6972 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:54:41.782553 30840 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff5039613fa14a0fa43b9dcffbff6972 [term 1 LEADER]: Becoming Leader. State: Replica: ff5039613fa14a0fa43b9dcffbff6972, State: Running, Role: LEADER
I20260809 08:54:41.782553 30836 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff5039613fa14a0fa43b9dcffbff6972 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:54:41.782672 30840 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff5039613fa14a0fa43b9dcffbff6972 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff5039613fa14a0fa43b9dcffbff6972" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 39957 } }
I20260809 08:54:41.783066 30841 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff5039613fa14a0fa43b9dcffbff6972 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff5039613fa14a0fa43b9dcffbff6972" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff5039613fa14a0fa43b9dcffbff6972" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 39957 } } }
I20260809 08:54:41.783161 30841 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff5039613fa14a0fa43b9dcffbff6972 [sys.catalog]: This master's current role is: LEADER
I20260809 08:54:41.783283 30844 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff5039613fa14a0fa43b9dcffbff6972 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff5039613fa14a0fa43b9dcffbff6972. Latest consensus state: current_term: 1 leader_uuid: "ff5039613fa14a0fa43b9dcffbff6972" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff5039613fa14a0fa43b9dcffbff6972" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 39957 } } }
I20260809 08:54:41.783401 30844 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff5039613fa14a0fa43b9dcffbff6972 [sys.catalog]: This master's current role is: LEADER
I20260809 08:54:41.783408 30854 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:54:41.783679 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 30742
I20260809 08:54:41.783797 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:54:41.784060 8808 external_mini_cluster.cc:1428] Setting key 8f4cffb15620bc62dc3b051dccd146ab
I20260809 08:54:41.784261 30854 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:54:41.785902 30854 catalog_manager.cc:1347] Generated new cluster ID: 5d5ffc1477fd45ab9ab1ba5d9f403af0
I20260809 08:54:41.785945 30854 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 08:54:41 dist-test-slave-t3q3 krb5kdc[30715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:54:41.799111 30868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:41.784685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39602 (local address 127.8.154.62:39957)
0809 08:54:41.788192 (+ 3507us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:41.788202 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:41.788236 (+ 34us) server_negotiation.cc:407] Connection header received
0809 08:54:41.788354 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:41.788361 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:41.788425 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:41.788570 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:54:41.789384 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:41.790782 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:41.791883 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:41.792536 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:41.792635 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:41.795451 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:54:41.795456 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:54:41.795456 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:54:41.795468 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:54:41.797460 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:41.797855 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:41.797858 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:41.797859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:41.797968 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:41.798173 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:41.798174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:41.798174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:41.798486 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:54:41.798612 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:41.798736 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 08:54:41.798875 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3448,"thread_start_us":87,"threads_started":1}
I20260809 08:54:41.800220 8808 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:54:41.806358 30854 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:54:41.806854 30854 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:54:41.811260 30854 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff5039613fa14a0fa43b9dcffbff6972: Generated new TSK 0
I20260809 08:54:41.811444 30854 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:54:41.854269 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 30742
2026-08-09T08:54:41Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1610 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:54:41 dist-test-slave-t3q3 krb5kdc[30877](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:54:41 dist-test-slave-t3q3 krb5kdc[30877](info): set up 2 sockets
Aug 09 08:54:41 dist-test-slave-t3q3 krb5kdc[30877](info): commencing operation
krb5kdc: starting...
W20260809 08:54:43.116539 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:54:43 dist-test-slave-t3q3 krb5kdc[30877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:54:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:54:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
I20260809 08:54:43.243793 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:37419
--webserver_interface=127.8.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:36087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:37419
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:54:43.327838 30895 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:54:43.328137 30895 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:54:43.328202 30895 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:54:43.329396 30895 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:54:43.329466 30895 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:54:43.329490 30895 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:54:43.329511 30895 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:54:43.329542 30895 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:54:43.331077 30895 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:36087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:37419
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:37419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30895
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:54:43.331382 30895 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:54:43.331630 30895 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:54:43.336887 30902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:43.336897 30903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:43.336897 30906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:54:43.337070 30895 server_base.cc:1034] running on GCE node
I20260809 08:54:43.337441 30895 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:54:43.337764 30895 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:54:43.338941 30895 hybrid_clock.cc:648] HybridClock initialized: now 1786265683338935 us; error 39 us; skew 500 ppm
Aug 09 08:54:43 dist-test-slave-t3q3 krb5kdc[30877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265683, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:54:43.341506 30895 init.cc:377] Logged in from keytab as kudu/127.8.154.62@KRBTEST.COM (short username kudu)
I20260809 08:54:43.341985 30895 webserver.cc:460] Webserver started at http://127.8.154.62:36443/ using document root <none> and password file <none>
I20260809 08:54:43.342207 30895 fs_manager.cc:359] Metadata directory not provided
I20260809 08:54:43.342276 30895 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:54:43.342371 30895 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:54:43.343407 30895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "b23f9ebbe3384808a243440a02c63001"
format_stamp: "Formatted at 2026-08-09 08:54:43 on dist-test-slave-t3q3"
server_key: "d312bb31f3d106be93c32960feaa9dfd"
server_key_iv: "1c91a7ab1a593286f68de47d151d6e0b"
server_key_version: "encryptionkey@0"
I20260809 08:54:43.343711 30895 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "b23f9ebbe3384808a243440a02c63001"
format_stamp: "Formatted at 2026-08-09 08:54:43 on dist-test-slave-t3q3"
server_key: "d312bb31f3d106be93c32960feaa9dfd"
server_key_iv: "1c91a7ab1a593286f68de47d151d6e0b"
server_key_version: "encryptionkey@0"
I20260809 08:54:43.345165 30895 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:54:43.345878 30919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:43.346076 30895 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:54:43.346169 30895 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "b23f9ebbe3384808a243440a02c63001"
format_stamp: "Formatted at 2026-08-09 08:54:43 on dist-test-slave-t3q3"
server_key: "d312bb31f3d106be93c32960feaa9dfd"
server_key_iv: "1c91a7ab1a593286f68de47d151d6e0b"
server_key_version: "encryptionkey@0"
I20260809 08:54:43.346228 30895 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:54:43.362457 30895 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:54:43.364863 30895 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:54:43.364984 30895 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:54:43.368942 30895 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:37419
I20260809 08:54:43.369050 30990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:37419 every 8 connection(s)
I20260809 08:54:43.369400 30895 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:54:43.370535 30991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:54:43.372922 30991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001: Bootstrap starting.
I20260809 08:54:43.373565 30991 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001: Neither blocks nor log segments found. Creating new log.
I20260809 08:54:43.373864 30991 log.cc:824] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001: Log is configured to *not* fsync() on all Append() calls
I20260809 08:54:43.374608 30991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001: No bootstrap required, opened a new log
I20260809 08:54:43.376258 30991 raft_consensus.cc:348] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b23f9ebbe3384808a243440a02c63001" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 37419 } }
I20260809 08:54:43.376423 30991 raft_consensus.cc:374] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:54:43.376475 30991 raft_consensus.cc:729] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b23f9ebbe3384808a243440a02c63001, State: Initialized, Role: FOLLOWER
I20260809 08:54:43.376645 30991 consensus_queue.cc:260] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b23f9ebbe3384808a243440a02c63001" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 37419 } }
I20260809 08:54:43.376748 30991 raft_consensus.cc:388] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:54:43.376792 30991 raft_consensus.cc:482] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:54:43.376845 30991 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:54:43.377643 30991 raft_consensus.cc:504] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b23f9ebbe3384808a243440a02c63001" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 37419 } }
I20260809 08:54:43.377766 30991 leader_election.cc:302] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b23f9ebbe3384808a243440a02c63001; no voters:
I20260809 08:54:43.377952 30991 leader_election.cc:288] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:54:43.377991 30995 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:54:43.378186 30995 raft_consensus.cc:686] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [term 1 LEADER]: Becoming Leader. State: Replica: b23f9ebbe3384808a243440a02c63001, State: Running, Role: LEADER
I20260809 08:54:43.378235 30991 sys_catalog.cc:564] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:54:43.378316 30995 consensus_queue.cc:237] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b23f9ebbe3384808a243440a02c63001" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 37419 } }
I20260809 08:54:43.378698 30996 sys_catalog.cc:455] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b23f9ebbe3384808a243440a02c63001" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b23f9ebbe3384808a243440a02c63001" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 37419 } } }
I20260809 08:54:43.378799 30996 sys_catalog.cc:458] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [sys.catalog]: This master's current role is: LEADER
I20260809 08:54:43.378703 30997 sys_catalog.cc:455] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b23f9ebbe3384808a243440a02c63001. Latest consensus state: current_term: 1 leader_uuid: "b23f9ebbe3384808a243440a02c63001" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b23f9ebbe3384808a243440a02c63001" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 37419 } } }
I20260809 08:54:43.378890 30997 sys_catalog.cc:458] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [sys.catalog]: This master's current role is: LEADER
I20260809 08:54:43.379061 31011 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:54:43.379558 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 30895
I20260809 08:54:43.379700 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:54:43.379803 31011 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:54:43.379940 8808 external_mini_cluster.cc:1428] Setting key f938911bd9fb2c94b9e9034ad480b7d7
I20260809 08:54:43.381633 31011 catalog_manager.cc:1347] Generated new cluster ID: 1e6951e211704a35ade02a0bb7a08b54
I20260809 08:54:43.381688 31011 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 08:54:43 dist-test-slave-t3q3 krb5kdc[30877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:54:43.390085 31011 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:54:43.390591 31025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:43.380454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52476 (local address 127.8.154.62:37419)
0809 08:54:43.380668 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:43.380678 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:43.380718 (+ 40us) server_negotiation.cc:407] Connection header received
0809 08:54:43.380878 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:43.380885 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:43.380941 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:43.381231 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:54:43.381910 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:43.382872 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:43.383681 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:43.384213 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:43.384334 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:43.386699 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:54:43.386703 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:54:43.386703 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:54:43.386715 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:54:43.388823 (+ 2108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:43.389223 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:43.389226 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:43.389227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:43.389339 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:43.389572 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:43.389573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:43.389573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:43.389845 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:54:43.390043 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:43.390243 (+ 200us) server_negotiation.cc:299] Negotiation successful
0809 08:54:43.390401 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":159,"thread_start_us":69,"threads_started":1}
I20260809 08:54:43.390772 31011 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:54:43.395821 31011 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001: Generated new TSK 0
I20260809 08:54:43.396039 31011 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab.
Entry for principal kudu/127.8.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab.
I20260809 08:54:43.430291 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.1:0
--local_ip_for_outbound_sockets=127.8.154.1
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:37419
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:36087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:54:43.518838 31029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:54:43.519083 31029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:54:43.519120 31029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:54:43.520344 31029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:54:43.520416 31029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:54:43.520459 31029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.1
I20260809 08:54:43.521955 31029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:36087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:37419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.1
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:54:43.522228 31029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:54:43.522472 31029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:54:43.528470 31040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:43.528442 31042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:43.528457 31039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:54:43.528627 31029 server_base.cc:1034] running on GCE node
I20260809 08:54:43.528898 31029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:54:43.529156 31029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:54:43.530319 31029 hybrid_clock.cc:648] HybridClock initialized: now 1786265683530300 us; error 38 us; skew 500 ppm
Aug 09 08:54:43 dist-test-slave-t3q3 krb5kdc[30877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265683, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:54:43.532863 31029 init.cc:377] Logged in from keytab as kudu/127.8.154.1@KRBTEST.COM (short username kudu)
I20260809 08:54:43.533320 31029 webserver.cc:460] Webserver started at http://127.8.154.1:34047/ using document root <none> and password file <none>
I20260809 08:54:43.533565 31029 fs_manager.cc:359] Metadata directory not provided
I20260809 08:54:43.533635 31029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:54:43.533787 31029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:54:43.534816 31029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/data/instance:
uuid: "4715c949a24b43a3a679777781f0c09b"
format_stamp: "Formatted at 2026-08-09 08:54:43 on dist-test-slave-t3q3"
server_key: "d76fc1bc985111a550198fab0c5f36a8"
server_key_iv: "7f3993954bbffad082306b0d14120418"
server_key_version: "encryptionkey@0"
I20260809 08:54:43.535180 31029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance:
uuid: "4715c949a24b43a3a679777781f0c09b"
format_stamp: "Formatted at 2026-08-09 08:54:43 on dist-test-slave-t3q3"
server_key: "d76fc1bc985111a550198fab0c5f36a8"
server_key_iv: "7f3993954bbffad082306b0d14120418"
server_key_version: "encryptionkey@0"
I20260809 08:54:43.536659 31029 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:54:43.537396 31050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:43.537560 31029 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:54:43.537660 31029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/wal
uuid: "4715c949a24b43a3a679777781f0c09b"
format_stamp: "Formatted at 2026-08-09 08:54:43 on dist-test-slave-t3q3"
server_key: "d76fc1bc985111a550198fab0c5f36a8"
server_key_iv: "7f3993954bbffad082306b0d14120418"
server_key_version: "encryptionkey@0"
I20260809 08:54:43.537743 31029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:54:43.553030 31029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:54:43.554940 31029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:54:43.555115 31029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:54:43.555349 31029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:54:43.555665 31029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:54:43.555718 31029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:43.555761 31029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:54:43.555788 31029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:43.561362 31029 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.1:43327
I20260809 08:54:43.561443 31225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.1:43327 every 8 connection(s)
I20260809 08:54:43.561775 31029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
I20260809 08:54:43.566494 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 31029
I20260809 08:54:43.566627 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance
I20260809 08:54:43.567137 8808 external_mini_cluster.cc:1428] Setting key fd45eb96b27b3b8f7a33a58126751c82
Aug 09 08:54:43 dist-test-slave-t3q3 krb5kdc[30877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265683, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:54:43.574271 31232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:43.562550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.1:42705)
0809 08:54:43.562718 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:43.562760 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 08:54:43.563246 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:54:43.563470 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:54:43.563473 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:54:43.564126 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:54:43.564333 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:43.564338 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:43.565010 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:43.565011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:43.565817 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:43.565819 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:43.566234 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:43.566236 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:43.566441 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:43.569633 (+ 3192us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:54:43.569644 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:54:43.571015 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:54:43.572909 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:43.572916 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:43.572918 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:43.573162 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:43.573388 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:43.573388 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:43.573388 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:43.573484 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:43.573781 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:54:43.573786 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:54:43.573898 (+ 112us) client_negotiation.cc:769] Sending connection context
0809 08:54:43.573976 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 08:54:43.574111 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":98,"threads_started":1}
I20260809 08:54:43.574281 31025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:43.562615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:42705 (local address 127.8.154.62:37419)
0809 08:54:43.562713 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:43.562715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:43.562905 (+ 190us) server_negotiation.cc:407] Connection header received
0809 08:54:43.563331 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:43.563332 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:43.563348 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:43.563415 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:54:43.564423 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:43.564926 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:43.565913 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:43.566141 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:43.566206 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:43.571115 (+ 4909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:54:43.571117 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:54:43.571118 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:54:43.571127 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:54:43.572830 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:43.573241 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:43.573242 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:43.573243 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:43.573311 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:43.573548 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:43.573549 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:43.573550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:43.573705 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:54:43.573780 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:43.574028 (+ 248us) server_negotiation.cc:299] Negotiation successful
0809 08:54:43.574140 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 08:54:43.574913 31227 heartbeater.cc:347] Connected to a master server at 127.8.154.62:37419
I20260809 08:54:43.575011 31227 heartbeater.cc:464] Registering TS with master...
I20260809 08:54:43.575214 31227 heartbeater.cc:511] Master 127.8.154.62:37419 requested a full tablet report, sending...
I20260809 08:54:43.575822 30945 ts_manager.cc:194] Registered new tserver with Master: 4715c949a24b43a3a679777781f0c09b (127.8.154.1:43327)
I20260809 08:54:43.576813 30945 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.1@KRBTEST.COM'} at 127.8.154.1:42705
WARNING: no policy specified for kudu/127.8.154.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab.
Entry for principal kudu/127.8.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab.
I20260809 08:54:43.595667 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.2:0
--local_ip_for_outbound_sockets=127.8.154.2
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:37419
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:36087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:54:43.679433 31236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:54:43.679682 31236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:54:43.679706 31236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:54:43.680863 31236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:54:43.680907 31236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:54:43.680932 31236 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.2
I20260809 08:54:43.682384 31236 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:36087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:37419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.2
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:54:43.682641 31236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:54:43.682879 31236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:54:43.687984 31252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:43.687984 31250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:43.688050 31246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:54:43.688318 31236 server_base.cc:1034] running on GCE node
I20260809 08:54:43.688526 31236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:54:43.688804 31236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:54:43.689957 31236 hybrid_clock.cc:648] HybridClock initialized: now 1786265683689947 us; error 35 us; skew 500 ppm
Aug 09 08:54:43 dist-test-slave-t3q3 krb5kdc[30877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265683, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:54:43.692619 31236 init.cc:377] Logged in from keytab as kudu/127.8.154.2@KRBTEST.COM (short username kudu)
I20260809 08:54:43.693118 31236 webserver.cc:460] Webserver started at http://127.8.154.2:38757/ using document root <none> and password file <none>
I20260809 08:54:43.693370 31236 fs_manager.cc:359] Metadata directory not provided
I20260809 08:54:43.693418 31236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:54:43.693576 31236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:54:43.694636 31236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/data/instance:
uuid: "0201a5d9a2104dc397da46d707c50b73"
format_stamp: "Formatted at 2026-08-09 08:54:43 on dist-test-slave-t3q3"
server_key: "1ffd114d8b75f47462163f7358c26adf"
server_key_iv: "11db7ede9c150dc23c774fa4925a19d0"
server_key_version: "encryptionkey@0"
I20260809 08:54:43.694993 31236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance:
uuid: "0201a5d9a2104dc397da46d707c50b73"
format_stamp: "Formatted at 2026-08-09 08:54:43 on dist-test-slave-t3q3"
server_key: "1ffd114d8b75f47462163f7358c26adf"
server_key_iv: "11db7ede9c150dc23c774fa4925a19d0"
server_key_version: "encryptionkey@0"
I20260809 08:54:43.696388 31236 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 08:54:43.697155 31266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:43.697342 31236 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:54:43.697435 31236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/wal
uuid: "0201a5d9a2104dc397da46d707c50b73"
format_stamp: "Formatted at 2026-08-09 08:54:43 on dist-test-slave-t3q3"
server_key: "1ffd114d8b75f47462163f7358c26adf"
server_key_iv: "11db7ede9c150dc23c774fa4925a19d0"
server_key_version: "encryptionkey@0"
I20260809 08:54:43.697510 31236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:54:43.709893 31236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:54:43.711869 31236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:54:43.712041 31236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:54:43.712342 31236 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:54:43.712656 31236 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:54:43.712692 31236 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:43.712742 31236 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:54:43.712774 31236 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:43.718396 31236 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.2:43885
I20260809 08:54:43.718505 31441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.2:43885 every 8 connection(s)
I20260809 08:54:43.718806 31236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
I20260809 08:54:43.721549 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 31236
I20260809 08:54:43.721668 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance
I20260809 08:54:43.721843 8808 external_mini_cluster.cc:1428] Setting key 35d73b67a15fde5e483c155972e840f5
Aug 09 08:54:43 dist-test-slave-t3q3 krb5kdc[30877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265683, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:54:43.729378 31025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:43.719578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:44109 (local address 127.8.154.62:37419)
0809 08:54:43.719668 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:43.719671 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:43.719863 (+ 192us) server_negotiation.cc:407] Connection header received
0809 08:54:43.720406 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:43.720407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:43.720418 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:43.720472 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:54:43.721383 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:43.722155 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:43.723702 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:43.723944 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:43.724028 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:43.726341 (+ 2313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:54:43.726343 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:54:43.726343 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:54:43.726351 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:54:43.727832 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:43.728360 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:43.728361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:43.728362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:43.728436 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:43.728707 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:43.728708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:43.728708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:43.728827 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:54:43.728905 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:43.729198 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 08:54:43.729285 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 08:54:43.729564 31447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:43.719532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.2:44109)
0809 08:54:43.719738 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:43.719774 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 08:54:43.720313 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:54:43.720526 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:54:43.720529 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:54:43.721093 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:54:43.721282 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:43.721287 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:43.722293 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:43.722294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:43.723606 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:43.723611 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:43.724043 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:43.724044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:43.724243 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:43.724847 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:54:43.724855 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:54:43.726238 (+ 1383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:54:43.727940 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:43.727947 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:43.727949 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:43.728264 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:43.728509 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:43.728510 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:43.728510 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:43.728629 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:43.728897 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:54:43.728904 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:54:43.729054 (+ 150us) client_negotiation.cc:769] Sending connection context
0809 08:54:43.729162 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 08:54:43.729328 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":127,"threads_started":1}
I20260809 08:54:43.730053 31442 heartbeater.cc:347] Connected to a master server at 127.8.154.62:37419
I20260809 08:54:43.730154 31442 heartbeater.cc:464] Registering TS with master...
I20260809 08:54:43.730358 31442 heartbeater.cc:511] Master 127.8.154.62:37419 requested a full tablet report, sending...
I20260809 08:54:43.730765 30945 ts_manager.cc:194] Registered new tserver with Master: 0201a5d9a2104dc397da46d707c50b73 (127.8.154.2:43885)
I20260809 08:54:43.731320 30945 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.2@KRBTEST.COM'} at 127.8.154.2:44109
WARNING: no policy specified for kudu/127.8.154.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab.
Entry for principal kudu/127.8.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab.
I20260809 08:54:43.750355 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.3:0
--local_ip_for_outbound_sockets=127.8.154.3
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:37419
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:36087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:54:43.832746 31451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:54:43.832966 31451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:54:43.833009 31451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:54:43.834196 31451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:54:43.834240 31451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:54:43.834295 31451 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.3
I20260809 08:54:43.835748 31451 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:36087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:37419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.3
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:54:43.835985 31451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:54:43.836297 31451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:54:43.842423 31460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:43.842418 31462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:54:43.842645 31451 server_base.cc:1034] running on GCE node
W20260809 08:54:43.842679 31459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:54:43.842962 31451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:54:43.843240 31451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:54:43.844398 31451 hybrid_clock.cc:648] HybridClock initialized: now 1786265683844378 us; error 40 us; skew 500 ppm
Aug 09 08:54:43 dist-test-slave-t3q3 krb5kdc[30877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265683, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:54:43.847061 31451 init.cc:377] Logged in from keytab as kudu/127.8.154.3@KRBTEST.COM (short username kudu)
I20260809 08:54:43.847606 31451 webserver.cc:460] Webserver started at http://127.8.154.3:46567/ using document root <none> and password file <none>
I20260809 08:54:43.847839 31451 fs_manager.cc:359] Metadata directory not provided
I20260809 08:54:43.847910 31451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:54:43.848119 31451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:54:43.849255 31451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/data/instance:
uuid: "8561daa539d243618585ad07d97c21cb"
format_stamp: "Formatted at 2026-08-09 08:54:43 on dist-test-slave-t3q3"
server_key: "0d8a25b2dda597ccbd3c810d13cce2d2"
server_key_iv: "23abf76242ce79a756dde566f53d4e9b"
server_key_version: "encryptionkey@0"
I20260809 08:54:43.849640 31451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance:
uuid: "8561daa539d243618585ad07d97c21cb"
format_stamp: "Formatted at 2026-08-09 08:54:43 on dist-test-slave-t3q3"
server_key: "0d8a25b2dda597ccbd3c810d13cce2d2"
server_key_iv: "23abf76242ce79a756dde566f53d4e9b"
server_key_version: "encryptionkey@0"
I20260809 08:54:43.851030 31451 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:54:43.851825 31476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:43.851997 31451 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:54:43.852147 31451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/wal
uuid: "8561daa539d243618585ad07d97c21cb"
format_stamp: "Formatted at 2026-08-09 08:54:43 on dist-test-slave-t3q3"
server_key: "0d8a25b2dda597ccbd3c810d13cce2d2"
server_key_iv: "23abf76242ce79a756dde566f53d4e9b"
server_key_version: "encryptionkey@0"
I20260809 08:54:43.852239 31451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:54:43.876555 31451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:54:43.878849 31451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:54:43.879019 31451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:54:43.879294 31451 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:54:43.879670 31451 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:54:43.879719 31451 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:43.879760 31451 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:54:43.879787 31451 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:43.886143 31451 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.3:35211
I20260809 08:54:43.886183 31645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.3:35211 every 8 connection(s)
I20260809 08:54:43.886576 31451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
Aug 09 08:54:43 dist-test-slave-t3q3 krb5kdc[30877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265683, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:54:43.895632 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 31451
I20260809 08:54:43.895725 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance
I20260809 08:54:43.895918 8808 external_mini_cluster.cc:1428] Setting key 27a00f98f78fbde69716ab2739e6c8f8
I20260809 08:54:43.897090 31025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:43.887375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:34963 (local address 127.8.154.62:37419)
0809 08:54:43.887449 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:43.887452 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:43.887668 (+ 216us) server_negotiation.cc:407] Connection header received
0809 08:54:43.888229 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:43.888230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:43.888240 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:43.888282 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:54:43.889270 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:43.889784 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:43.890795 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:43.891115 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:43.891190 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:43.894037 (+ 2847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:54:43.894039 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:54:43.894039 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:54:43.894047 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:54:43.895366 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:43.896030 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:43.896031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:43.896031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:43.896120 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:43.896363 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:43.896364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:43.896364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:43.896511 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:54:43.896602 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:43.896910 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 08:54:43.896993 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 08:54:43.897315 31650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:43.887363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.3:34963)
0809 08:54:43.887538 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:43.887576 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 08:54:43.888114 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:54:43.888343 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:54:43.888347 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:54:43.888957 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:54:43.889164 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:43.889169 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:43.889873 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:43.889874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:43.890689 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:43.890690 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:43.891204 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:43.891204 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:43.891306 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:43.891916 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:54:43.891926 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:54:43.893941 (+ 2015us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:54:43.895483 (+ 1542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:43.895490 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:43.895493 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:43.895836 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:43.896208 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:43.896209 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:43.896209 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:43.896314 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:43.896607 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:54:43.896615 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:54:43.896776 (+ 161us) client_negotiation.cc:769] Sending connection context
0809 08:54:43.896889 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 08:54:43.897070 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":93,"threads_started":1}
I20260809 08:54:43.897786 31648 heartbeater.cc:347] Connected to a master server at 127.8.154.62:37419
I20260809 08:54:43.897891 31648 heartbeater.cc:464] Registering TS with master...
I20260809 08:54:43.898082 31648 heartbeater.cc:511] Master 127.8.154.62:37419 requested a full tablet report, sending...
I20260809 08:54:43.898468 30945 ts_manager.cc:194] Registered new tserver with Master: 8561daa539d243618585ad07d97c21cb (127.8.154.3:35211)
I20260809 08:54:43.898805 30945 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.3@KRBTEST.COM'} at 127.8.154.3:34963
I20260809 08:54:43.907823 8808 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:54:43.915706 31025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:43.909294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52490 (local address 127.8.154.62:37419)
0809 08:54:43.909367 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:43.909369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:43.909468 (+ 99us) server_negotiation.cc:407] Connection header received
0809 08:54:43.909551 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:43.909552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:43.909568 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:43.909630 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:54:43.910252 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:43.910738 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:43.911473 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:43.911669 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:43.911746 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:43.913074 (+ 1328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:54:43.913076 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:54:43.913076 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:54:43.913083 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:54:43.914386 (+ 1303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:43.914796 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:43.914803 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:43.914804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:43.914870 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:43.915081 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:43.915082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:43.915083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:43.915245 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:54:43.915348 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:43.915508 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 08:54:43.915601 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 08:54:43.917479 30945 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52490:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:54:43.919028 30945 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:54:43.925874 31664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:43.921546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:43885 (local address 127.0.0.1:56660)
0809 08:54:43.921709 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:43.921750 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 08:54:43.921817 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:54:43.922265 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:54:43.922268 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:54:43.922271 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:54:43.922347 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:43.922350 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:43.923348 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:43.923349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:43.924633 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:43.924636 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:43.925543 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:43.925544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:43.925632 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:43.925649 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 08:54:43.925723 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 08:54:43.925776 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":52,"threads_started":1}
I20260809 08:54:43.925894 31668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:43.921649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56660 (local address 127.8.154.2:43885)
0809 08:54:43.921969 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:43.921975 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:43.921984 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:54:43.922027 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:43.922029 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:43.922044 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:43.922237 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:54:43.922418 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:43.923238 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:43.924728 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:43.925449 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:43.925525 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:43.925584 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:43.925752 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 08:54:43.925805 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":99,"threads_started":1}
I20260809 08:54:43.926065 31670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:43.921864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34720 (local address 127.8.154.3:35211)
0809 08:54:43.922071 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:43.922083 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:43.922299 (+ 216us) server_negotiation.cc:407] Connection header received
0809 08:54:43.922346 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:43.922348 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:43.922366 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:43.922439 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:54:43.922661 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:43.923761 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:43.925041 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:43.925555 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:43.925633 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:43.925689 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:43.925921 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 08:54:43.925970 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":106,"threads_started":1}
I20260809 08:54:43.926223 31665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:43.921552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:35211 (local address 127.0.0.1:34720)
0809 08:54:43.922179 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:43.922189 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:54:43.922241 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:54:43.922527 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:54:43.922528 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:54:43.922529 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:54:43.922579 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:43.922581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:43.923879 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:43.923880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:43.924946 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:43.924948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:43.925725 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:43.925726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:43.925781 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:43.925783 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 08:54:43.925831 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 08:54:43.925863 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"mutex_wait_us":1,"thread_start_us":79,"threads_started":1}
I20260809 08:54:43.926501 31669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:43.921755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:43327 (local address 127.0.0.1:60122)
0809 08:54:43.922347 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:43.922357 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:54:43.922407 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:54:43.922635 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:54:43.922636 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:54:43.922638 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:54:43.922683 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:43.922685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:43.923911 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:43.923912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:43.924964 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:43.924966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:43.925910 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:43.925912 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:43.926332 (+ 420us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:43.926338 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 08:54:43.926397 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 08:54:43.926435 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":83,"threads_started":1}
I20260809 08:54:43.926551 31672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:43.921897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60122 (local address 127.8.154.1:43327)
0809 08:54:43.922110 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:43.922113 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:43.922473 (+ 360us) server_negotiation.cc:407] Connection header received
0809 08:54:43.922526 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:43.922529 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:43.922544 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:43.922624 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:54:43.922756 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:43.923802 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:43.925058 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:43.925758 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:43.925822 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:43.925871 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:43.926427 (+ 556us) server_negotiation.cc:299] Negotiation successful
0809 08:54:43.926471 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":100,"threads_started":1}
I20260809 08:54:43.926708 31341 tablet_service.cc:1467] Processing CreateTablet for tablet 078d3f7bff294fca8751a6d153e4e5aa (DEFAULT_TABLE table=test-table [id=65842aa01fcc468d9bbf3cc5aa45efc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:54:43.926893 31551 tablet_service.cc:1467] Processing CreateTablet for tablet 078d3f7bff294fca8751a6d153e4e5aa (DEFAULT_TABLE table=test-table [id=65842aa01fcc468d9bbf3cc5aa45efc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:54:43.927109 31341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 078d3f7bff294fca8751a6d153e4e5aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:54:43.927157 31551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 078d3f7bff294fca8751a6d153e4e5aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:54:43.927167 31124 tablet_service.cc:1467] Processing CreateTablet for tablet 078d3f7bff294fca8751a6d153e4e5aa (DEFAULT_TABLE table=test-table [id=65842aa01fcc468d9bbf3cc5aa45efc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:54:43.927461 31124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 078d3f7bff294fca8751a6d153e4e5aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:54:43.929642 31674 tablet_bootstrap.cc:492] T 078d3f7bff294fca8751a6d153e4e5aa P 8561daa539d243618585ad07d97c21cb: Bootstrap starting.
I20260809 08:54:43.930256 31674 tablet_bootstrap.cc:654] T 078d3f7bff294fca8751a6d153e4e5aa P 8561daa539d243618585ad07d97c21cb: Neither blocks nor log segments found. Creating new log.
I20260809 08:54:43.930554 31676 tablet_bootstrap.cc:492] T 078d3f7bff294fca8751a6d153e4e5aa P 0201a5d9a2104dc397da46d707c50b73: Bootstrap starting.
I20260809 08:54:43.930794 31677 tablet_bootstrap.cc:492] T 078d3f7bff294fca8751a6d153e4e5aa P 4715c949a24b43a3a679777781f0c09b: Bootstrap starting.
I20260809 08:54:43.931252 31676 tablet_bootstrap.cc:654] T 078d3f7bff294fca8751a6d153e4e5aa P 0201a5d9a2104dc397da46d707c50b73: Neither blocks nor log segments found. Creating new log.
I20260809 08:54:43.931252 31674 log.cc:824] T 078d3f7bff294fca8751a6d153e4e5aa P 8561daa539d243618585ad07d97c21cb: Log is configured to *not* fsync() on all Append() calls
I20260809 08:54:43.931469 31677 tablet_bootstrap.cc:654] T 078d3f7bff294fca8751a6d153e4e5aa P 4715c949a24b43a3a679777781f0c09b: Neither blocks nor log segments found. Creating new log.
I20260809 08:54:43.931864 31676 log.cc:824] T 078d3f7bff294fca8751a6d153e4e5aa P 0201a5d9a2104dc397da46d707c50b73: Log is configured to *not* fsync() on all Append() calls
I20260809 08:54:43.932181 31674 tablet_bootstrap.cc:492] T 078d3f7bff294fca8751a6d153e4e5aa P 8561daa539d243618585ad07d97c21cb: No bootstrap required, opened a new log
I20260809 08:54:43.932155 31677 log.cc:824] T 078d3f7bff294fca8751a6d153e4e5aa P 4715c949a24b43a3a679777781f0c09b: Log is configured to *not* fsync() on all Append() calls
I20260809 08:54:43.932272 31674 ts_tablet_manager.cc:1397] T 078d3f7bff294fca8751a6d153e4e5aa P 8561daa539d243618585ad07d97c21cb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 08:54:43.932600 31676 tablet_bootstrap.cc:492] T 078d3f7bff294fca8751a6d153e4e5aa P 0201a5d9a2104dc397da46d707c50b73: No bootstrap required, opened a new log
I20260809 08:54:43.932682 31676 ts_tablet_manager.cc:1397] T 078d3f7bff294fca8751a6d153e4e5aa P 0201a5d9a2104dc397da46d707c50b73: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 08:54:43.933032 31677 tablet_bootstrap.cc:492] T 078d3f7bff294fca8751a6d153e4e5aa P 4715c949a24b43a3a679777781f0c09b: No bootstrap required, opened a new log
I20260809 08:54:43.933100 31677 ts_tablet_manager.cc:1397] T 078d3f7bff294fca8751a6d153e4e5aa P 4715c949a24b43a3a679777781f0c09b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 08:54:43.934298 31674 raft_consensus.cc:348] T 078d3f7bff294fca8751a6d153e4e5aa P 8561daa539d243618585ad07d97c21cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8561daa539d243618585ad07d97c21cb" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 35211 } } peers { permanent_uuid: "0201a5d9a2104dc397da46d707c50b73" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43885 } } peers { permanent_uuid: "4715c949a24b43a3a679777781f0c09b" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43327 } }
I20260809 08:54:43.934469 31674 raft_consensus.cc:374] T 078d3f7bff294fca8751a6d153e4e5aa P 8561daa539d243618585ad07d97c21cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:54:43.934514 31674 raft_consensus.cc:729] T 078d3f7bff294fca8751a6d153e4e5aa P 8561daa539d243618585ad07d97c21cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8561daa539d243618585ad07d97c21cb, State: Initialized, Role: FOLLOWER
I20260809 08:54:43.934664 31674 consensus_queue.cc:260] T 078d3f7bff294fca8751a6d153e4e5aa P 8561daa539d243618585ad07d97c21cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8561daa539d243618585ad07d97c21cb" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 35211 } } peers { permanent_uuid: "0201a5d9a2104dc397da46d707c50b73" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43885 } } peers { permanent_uuid: "4715c949a24b43a3a679777781f0c09b" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43327 } }
I20260809 08:54:43.934690 31676 raft_consensus.cc:348] T 078d3f7bff294fca8751a6d153e4e5aa P 0201a5d9a2104dc397da46d707c50b73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8561daa539d243618585ad07d97c21cb" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 35211 } } peers { permanent_uuid: "0201a5d9a2104dc397da46d707c50b73" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43885 } } peers { permanent_uuid: "4715c949a24b43a3a679777781f0c09b" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43327 } }
I20260809 08:54:43.934877 31676 raft_consensus.cc:374] T 078d3f7bff294fca8751a6d153e4e5aa P 0201a5d9a2104dc397da46d707c50b73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:54:43.934923 31676 raft_consensus.cc:729] T 078d3f7bff294fca8751a6d153e4e5aa P 0201a5d9a2104dc397da46d707c50b73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0201a5d9a2104dc397da46d707c50b73, State: Initialized, Role: FOLLOWER
I20260809 08:54:43.934957 31674 ts_tablet_manager.cc:1428] T 078d3f7bff294fca8751a6d153e4e5aa P 8561daa539d243618585ad07d97c21cb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:54:43.935075 31648 heartbeater.cc:503] Master 127.8.154.62:37419 was elected leader, sending a full tablet report...
I20260809 08:54:43.935081 31676 consensus_queue.cc:260] T 078d3f7bff294fca8751a6d153e4e5aa P 0201a5d9a2104dc397da46d707c50b73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8561daa539d243618585ad07d97c21cb" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 35211 } } peers { permanent_uuid: "0201a5d9a2104dc397da46d707c50b73" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43885 } } peers { permanent_uuid: "4715c949a24b43a3a679777781f0c09b" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43327 } }
I20260809 08:54:43.935019 31677 raft_consensus.cc:348] T 078d3f7bff294fca8751a6d153e4e5aa P 4715c949a24b43a3a679777781f0c09b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8561daa539d243618585ad07d97c21cb" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 35211 } } peers { permanent_uuid: "0201a5d9a2104dc397da46d707c50b73" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43885 } } peers { permanent_uuid: "4715c949a24b43a3a679777781f0c09b" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43327 } }
I20260809 08:54:43.935179 31677 raft_consensus.cc:374] T 078d3f7bff294fca8751a6d153e4e5aa P 4715c949a24b43a3a679777781f0c09b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:54:43.935228 31677 raft_consensus.cc:729] T 078d3f7bff294fca8751a6d153e4e5aa P 4715c949a24b43a3a679777781f0c09b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4715c949a24b43a3a679777781f0c09b, State: Initialized, Role: FOLLOWER
I20260809 08:54:43.935426 31442 heartbeater.cc:503] Master 127.8.154.62:37419 was elected leader, sending a full tablet report...
I20260809 08:54:43.935482 31676 ts_tablet_manager.cc:1428] T 078d3f7bff294fca8751a6d153e4e5aa P 0201a5d9a2104dc397da46d707c50b73: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 08:54:43.935353 31677 consensus_queue.cc:260] T 078d3f7bff294fca8751a6d153e4e5aa P 4715c949a24b43a3a679777781f0c09b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8561daa539d243618585ad07d97c21cb" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 35211 } } peers { permanent_uuid: "0201a5d9a2104dc397da46d707c50b73" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43885 } } peers { permanent_uuid: "4715c949a24b43a3a679777781f0c09b" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43327 } }
I20260809 08:54:43.935930 31677 ts_tablet_manager.cc:1428] T 078d3f7bff294fca8751a6d153e4e5aa P 4715c949a24b43a3a679777781f0c09b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 08:54:43.936116 31227 heartbeater.cc:503] Master 127.8.154.62:37419 was elected leader, sending a full tablet report...
W20260809 08:54:43.969400 31444 tablet.cc:2369] T 078d3f7bff294fca8751a6d153e4e5aa P 0201a5d9a2104dc397da46d707c50b73: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:54:44.057396 31683 raft_consensus.cc:482] T 078d3f7bff294fca8751a6d153e4e5aa P 8561daa539d243618585ad07d97c21cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:54:44.057529 31683 raft_consensus.cc:504] T 078d3f7bff294fca8751a6d153e4e5aa P 8561daa539d243618585ad07d97c21cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8561daa539d243618585ad07d97c21cb" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 35211 } } peers { permanent_uuid: "0201a5d9a2104dc397da46d707c50b73" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43885 } } peers { permanent_uuid: "4715c949a24b43a3a679777781f0c09b" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43327 } }
I20260809 08:54:44.057999 31683 leader_election.cc:288] T 078d3f7bff294fca8751a6d153e4e5aa P 8561daa539d243618585ad07d97c21cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0201a5d9a2104dc397da46d707c50b73 (127.8.154.2:43885), 4715c949a24b43a3a679777781f0c09b (127.8.154.1:43327)
I20260809 08:54:44.060958 31668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:44.058289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:50637 (local address 127.8.154.2:43885)
0809 08:54:44.058360 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:44.058362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:44.058374 (+ 12us) server_negotiation.cc:407] Connection header received
0809 08:54:44.058402 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:44.058403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:44.058413 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:44.058498 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:54:44.058681 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.059171 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.060194 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.060577 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.060674 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:44.060685 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:44.060849 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 08:54:44.060886 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 08:54:44.060958 31650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:44.058197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:43885 (local address 127.8.154.3:50637)
0809 08:54:44.058251 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:44.058263 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:54:44.058332 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:54:44.058507 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:54:44.058509 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:54:44.058511 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:54:44.058598 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:44.058599 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.059261 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.059262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:44.059997 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:44.059998 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.060672 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.060673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:44.060762 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:44.060765 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:54:44.060842 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 08:54:44.060885 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 08:54:44.061268 31672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:44.058289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:41349 (local address 127.8.154.1:43327)
0809 08:54:44.058363 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:44.058365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:44.058662 (+ 297us) server_negotiation.cc:407] Connection header received
0809 08:54:44.058729 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:44.058731 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:44.058745 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:44.058819 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:54:44.058979 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.059452 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.060460 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.060834 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.060927 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:44.060936 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:44.061162 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 08:54:44.061207 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 08:54:44.061267 31687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:44.058204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:43327 (local address 127.8.154.3:41349)
0809 08:54:44.058582 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:44.058590 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 08:54:44.058668 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:54:44.058823 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:54:44.058823 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:54:44.058825 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:54:44.058904 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:44.058906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.059544 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.059545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:44.060370 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:44.060373 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.060928 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.060929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:44.061027 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:44.061032 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:54:44.061128 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 08:54:44.061178 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":181,"thread_start_us":134,"threads_started":1}
I20260809 08:54:44.061338 31372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "078d3f7bff294fca8751a6d153e4e5aa" candidate_uuid: "8561daa539d243618585ad07d97c21cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0201a5d9a2104dc397da46d707c50b73" is_pre_election: true
I20260809 08:54:44.061497 31372 raft_consensus.cc:2454] T 078d3f7bff294fca8751a6d153e4e5aa P 0201a5d9a2104dc397da46d707c50b73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8561daa539d243618585ad07d97c21cb in term 0.
I20260809 08:54:44.061650 31158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "078d3f7bff294fca8751a6d153e4e5aa" candidate_uuid: "8561daa539d243618585ad07d97c21cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4715c949a24b43a3a679777781f0c09b" is_pre_election: true
I20260809 08:54:44.061780 31477 leader_election.cc:302] T 078d3f7bff294fca8751a6d153e4e5aa P 8561daa539d243618585ad07d97c21cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0201a5d9a2104dc397da46d707c50b73, 8561daa539d243618585ad07d97c21cb; no voters:
I20260809 08:54:44.061825 31158 raft_consensus.cc:2454] T 078d3f7bff294fca8751a6d153e4e5aa P 4715c949a24b43a3a679777781f0c09b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8561daa539d243618585ad07d97c21cb in term 0.
I20260809 08:54:44.061941 31683 raft_consensus.cc:2781] T 078d3f7bff294fca8751a6d153e4e5aa P 8561daa539d243618585ad07d97c21cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:54:44.062022 31683 raft_consensus.cc:482] T 078d3f7bff294fca8751a6d153e4e5aa P 8561daa539d243618585ad07d97c21cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:54:44.062060 31683 raft_consensus.cc:3037] T 078d3f7bff294fca8751a6d153e4e5aa P 8561daa539d243618585ad07d97c21cb [term 0 FOLLOWER]: Advancing to term 1
W20260809 08:54:44.062461 31230 tablet.cc:2369] T 078d3f7bff294fca8751a6d153e4e5aa P 4715c949a24b43a3a679777781f0c09b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:54:44.062947 31683 raft_consensus.cc:504] T 078d3f7bff294fca8751a6d153e4e5aa P 8561daa539d243618585ad07d97c21cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8561daa539d243618585ad07d97c21cb" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 35211 } } peers { permanent_uuid: "0201a5d9a2104dc397da46d707c50b73" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43885 } } peers { permanent_uuid: "4715c949a24b43a3a679777781f0c09b" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43327 } }
I20260809 08:54:44.063097 31683 leader_election.cc:288] T 078d3f7bff294fca8751a6d153e4e5aa P 8561daa539d243618585ad07d97c21cb [CANDIDATE]: Term 1 election: Requested vote from peers 0201a5d9a2104dc397da46d707c50b73 (127.8.154.2:43885), 4715c949a24b43a3a679777781f0c09b (127.8.154.1:43327)
I20260809 08:54:44.063263 31158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "078d3f7bff294fca8751a6d153e4e5aa" candidate_uuid: "8561daa539d243618585ad07d97c21cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4715c949a24b43a3a679777781f0c09b"
I20260809 08:54:44.063269 31372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "078d3f7bff294fca8751a6d153e4e5aa" candidate_uuid: "8561daa539d243618585ad07d97c21cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0201a5d9a2104dc397da46d707c50b73"
I20260809 08:54:44.063344 31158 raft_consensus.cc:3037] T 078d3f7bff294fca8751a6d153e4e5aa P 4715c949a24b43a3a679777781f0c09b [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:54:44.063366 31372 raft_consensus.cc:3037] T 078d3f7bff294fca8751a6d153e4e5aa P 0201a5d9a2104dc397da46d707c50b73 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:54:44.064133 31372 raft_consensus.cc:2454] T 078d3f7bff294fca8751a6d153e4e5aa P 0201a5d9a2104dc397da46d707c50b73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8561daa539d243618585ad07d97c21cb in term 1.
I20260809 08:54:44.064193 31158 raft_consensus.cc:2454] T 078d3f7bff294fca8751a6d153e4e5aa P 4715c949a24b43a3a679777781f0c09b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8561daa539d243618585ad07d97c21cb in term 1.
I20260809 08:54:44.064328 31477 leader_election.cc:302] T 078d3f7bff294fca8751a6d153e4e5aa P 8561daa539d243618585ad07d97c21cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0201a5d9a2104dc397da46d707c50b73, 8561daa539d243618585ad07d97c21cb; no voters:
I20260809 08:54:44.064415 31683 raft_consensus.cc:2781] T 078d3f7bff294fca8751a6d153e4e5aa P 8561daa539d243618585ad07d97c21cb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:54:44.064594 31683 raft_consensus.cc:686] T 078d3f7bff294fca8751a6d153e4e5aa P 8561daa539d243618585ad07d97c21cb [term 1 LEADER]: Becoming Leader. State: Replica: 8561daa539d243618585ad07d97c21cb, State: Running, Role: LEADER
I20260809 08:54:44.064702 31683 consensus_queue.cc:237] T 078d3f7bff294fca8751a6d153e4e5aa P 8561daa539d243618585ad07d97c21cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8561daa539d243618585ad07d97c21cb" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 35211 } } peers { permanent_uuid: "0201a5d9a2104dc397da46d707c50b73" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43885 } } peers { permanent_uuid: "4715c949a24b43a3a679777781f0c09b" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43327 } }
I20260809 08:54:44.065521 30944 catalog_manager.cc:5495] T 078d3f7bff294fca8751a6d153e4e5aa P 8561daa539d243618585ad07d97c21cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 8561daa539d243618585ad07d97c21cb (127.8.154.3). New cstate: current_term: 1 leader_uuid: "8561daa539d243618585ad07d97c21cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8561daa539d243618585ad07d97c21cb" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 35211 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0201a5d9a2104dc397da46d707c50b73" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4715c949a24b43a3a679777781f0c09b" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43327 } health_report { overall_health: UNKNOWN } } }
I20260809 08:54:44.074421 31670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:44.072112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34734 (local address 127.8.154.3:35211)
0809 08:54:44.072168 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:44.072170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:44.072180 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:54:44.072211 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:44.072212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:44.072236 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:44.072312 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:54:44.072437 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.072878 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.073649 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.073840 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.073916 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:44.074095 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:54:44.074189 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:54:44.074280 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:44.074299 (+ 19us) server_negotiation.cc:299] Negotiation successful
0809 08:54:44.074318 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 08:54:44.076054 31158 raft_consensus.cc:1261] T 078d3f7bff294fca8751a6d153e4e5aa P 4715c949a24b43a3a679777781f0c09b [term 1 FOLLOWER]: Refusing update from remote peer 8561daa539d243618585ad07d97c21cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:54:44.076041 31372 raft_consensus.cc:1261] T 078d3f7bff294fca8751a6d153e4e5aa P 0201a5d9a2104dc397da46d707c50b73 [term 1 FOLLOWER]: Refusing update from remote peer 8561daa539d243618585ad07d97c21cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:54:44.076433 31690 consensus_queue.cc:1035] T 078d3f7bff294fca8751a6d153e4e5aa P 8561daa539d243618585ad07d97c21cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "0201a5d9a2104dc397da46d707c50b73" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:54:44.076563 31683 consensus_queue.cc:1035] T 078d3f7bff294fca8751a6d153e4e5aa P 8561daa539d243618585ad07d97c21cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "4715c949a24b43a3a679777781f0c09b" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:54:44.080166 31699 mvcc.cc:205] Tried to move back new op lower bound from 7316544241973170176 to 7316544241929576448. Current Snapshot: MvccSnapshot[applied={T|T < 7316544241973170176 or (T in {7316544241973170176})}]
I20260809 08:54:44.081296 30941 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:54:44.081384 30941 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:54:44.082109 30941 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001: Sending DeleteTablet for 3 replicas of tablet 078d3f7bff294fca8751a6d153e4e5aa
I20260809 08:54:44.082473 31551 tablet_service.cc:1514] Processing DeleteTablet for tablet 078d3f7bff294fca8751a6d153e4e5aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:54:44 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:34720
I20260809 08:54:44.082473 31341 tablet_service.cc:1514] Processing DeleteTablet for tablet 078d3f7bff294fca8751a6d153e4e5aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:54:44 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:56660
I20260809 08:54:44.082746 31709 tablet_replica.cc:330] T 078d3f7bff294fca8751a6d153e4e5aa P 0201a5d9a2104dc397da46d707c50b73: stopping tablet replica
I20260809 08:54:44.082739 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 31029
I20260809 08:54:44.082741 31124 tablet_service.cc:1514] Processing DeleteTablet for tablet 078d3f7bff294fca8751a6d153e4e5aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:54:44 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:60122
I20260809 08:54:44.096676 31708 tablet_replica.cc:330] T 078d3f7bff294fca8751a6d153e4e5aa P 8561daa539d243618585ad07d97c21cb: stopping tablet replica
I20260809 08:54:44.096814 31708 raft_consensus.cc:2229] T 078d3f7bff294fca8751a6d153e4e5aa P 8561daa539d243618585ad07d97c21cb [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:54:44.096927 31708 raft_consensus.cc:2258] T 078d3f7bff294fca8751a6d153e4e5aa P 8561daa539d243618585ad07d97c21cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:54:44.096989 31709 raft_consensus.cc:2229] T 078d3f7bff294fca8751a6d153e4e5aa P 0201a5d9a2104dc397da46d707c50b73 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:54:44.097142 31709 raft_consensus.cc:2258] T 078d3f7bff294fca8751a6d153e4e5aa P 0201a5d9a2104dc397da46d707c50b73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:54:44.097690 31709 ts_tablet_manager.cc:1905] T 078d3f7bff294fca8751a6d153e4e5aa P 0201a5d9a2104dc397da46d707c50b73: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:54:44.098338 31708 ts_tablet_manager.cc:1905] T 078d3f7bff294fca8751a6d153e4e5aa P 8561daa539d243618585ad07d97c21cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:54:44.099817 31709 ts_tablet_manager.cc:1918] T 078d3f7bff294fca8751a6d153e4e5aa P 0201a5d9a2104dc397da46d707c50b73: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:54:44.099905 31709 log.cc:1196] T 078d3f7bff294fca8751a6d153e4e5aa P 0201a5d9a2104dc397da46d707c50b73: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/wal/wals/078d3f7bff294fca8751a6d153e4e5aa
I20260809 08:54:44.100047 31708 ts_tablet_manager.cc:1918] T 078d3f7bff294fca8751a6d153e4e5aa P 8561daa539d243618585ad07d97c21cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:54:44.100178 31708 log.cc:1196] T 078d3f7bff294fca8751a6d153e4e5aa P 8561daa539d243618585ad07d97c21cb: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/wal/wals/078d3f7bff294fca8751a6d153e4e5aa
I20260809 08:54:44.100257 31709 ts_tablet_manager.cc:1939] T 078d3f7bff294fca8751a6d153e4e5aa P 0201a5d9a2104dc397da46d707c50b73: Deleting consensus metadata
I20260809 08:54:44.100520 31708 ts_tablet_manager.cc:1939] T 078d3f7bff294fca8751a6d153e4e5aa P 8561daa539d243618585ad07d97c21cb: Deleting consensus metadata
I20260809 08:54:44.100760 30920 catalog_manager.cc:4841] TS 0201a5d9a2104dc397da46d707c50b73 (127.8.154.2:43885): tablet 078d3f7bff294fca8751a6d153e4e5aa (table test-table [id=65842aa01fcc468d9bbf3cc5aa45efc5]) successfully deleted
I20260809 08:54:44.101081 30927 catalog_manager.cc:4841] TS 8561daa539d243618585ad07d97c21cb (127.8.154.3:35211): tablet 078d3f7bff294fca8751a6d153e4e5aa (table test-table [id=65842aa01fcc468d9bbf3cc5aa45efc5]) successfully deleted
W20260809 08:54:44.102476 30927 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.154.1:43327 (error 108)
I20260809 08:54:44.102725 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 31236
I20260809 08:54:44.103317 31669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:44.102887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:43327 (local address 127.0.0.1:60138)
0809 08:54:44.103248 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:44.103286 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.1:43327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338}
W20260809 08:54:44.105831 30927 catalog_manager.cc:4584] TS 4715c949a24b43a3a679777781f0c09b (127.8.154.1:43327): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 078d3f7bff294fca8751a6d153e4e5aa: Network error: Client connection negotiation failed: client connection to 127.8.154.1:43327: connect: Connection refused (error 111)
I20260809 08:54:44.119059 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 31451
I20260809 08:54:44.133251 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 30895
I20260809 08:54:44.143075 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:37419
--webserver_interface=127.8.154.62
--webserver_port=36443
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:36087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:37419
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:54:44.227458 31712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:54:44.227674 31712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:54:44.227698 31712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:54:44.228847 31712 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:54:44.228889 31712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:54:44.228904 31712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:54:44.228916 31712 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:54:44.228928 31712 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:54:44.230332 31712 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:36087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:37419
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:37419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=36443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:54:44.230526 31712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:54:44.230705 31712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:54:44.240357 31721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:44.240322 31724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:44.240322 31720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:54:44.240389 31712 server_base.cc:1034] running on GCE node
I20260809 08:54:44.240917 31712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:54:44.241256 31712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:54:44.242432 31712 hybrid_clock.cc:648] HybridClock initialized: now 1786265684242429 us; error 38 us; skew 500 ppm
Aug 09 08:54:44 dist-test-slave-t3q3 krb5kdc[30877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265684, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:54:44.245261 31712 init.cc:377] Logged in from keytab as kudu/127.8.154.62@KRBTEST.COM (short username kudu)
I20260809 08:54:44.245754 31712 webserver.cc:460] Webserver started at http://127.8.154.62:36443/ using document root <none> and password file <none>
I20260809 08:54:44.246016 31712 fs_manager.cc:359] Metadata directory not provided
I20260809 08:54:44.246101 31712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:54:44.247457 31712 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:54:44.248276 31733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:44.248454 31712 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:54:44.248564 31712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "b23f9ebbe3384808a243440a02c63001"
format_stamp: "Formatted at 2026-08-09 08:54:43 on dist-test-slave-t3q3"
server_key: "d312bb31f3d106be93c32960feaa9dfd"
server_key_iv: "1c91a7ab1a593286f68de47d151d6e0b"
server_key_version: "encryptionkey@0"
I20260809 08:54:44.248952 31712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:54:44.262789 31712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:54:44.265174 31712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:54:44.265345 31712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:54:44.269613 31712 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:37419
I20260809 08:54:44.269649 31816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:37419 every 8 connection(s)
I20260809 08:54:44.270069 31712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:54:44.271759 31817 sys_catalog.cc:263] Verifying existing consensus state
I20260809 08:54:44.272473 31817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001: Bootstrap starting.
I20260809 08:54:44.278519 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 31712
I20260809 08:54:44.278895 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.1:43327
--local_ip_for_outbound_sockets=127.8.154.1
--tserver_master_addrs=127.8.154.62:37419
--webserver_port=34047
--webserver_interface=127.8.154.1
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:36087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:54:44.281745 31817 log.cc:824] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001: Log is configured to *not* fsync() on all Append() calls
I20260809 08:54:44.283905 31817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 08:54:44.284281 31817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001: Bootstrap complete.
I20260809 08:54:44.286767 31817 raft_consensus.cc:348] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b23f9ebbe3384808a243440a02c63001" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 37419 } }
I20260809 08:54:44.286998 31817 raft_consensus.cc:729] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b23f9ebbe3384808a243440a02c63001, State: Initialized, Role: FOLLOWER
I20260809 08:54:44.287181 31817 consensus_queue.cc:260] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b23f9ebbe3384808a243440a02c63001" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 37419 } }
I20260809 08:54:44.287246 31817 raft_consensus.cc:388] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:54:44.287313 31817 raft_consensus.cc:482] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:54:44.287387 31817 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [term 1 FOLLOWER]: Advancing to term 2
I20260809 08:54:44.288555 31817 raft_consensus.cc:504] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b23f9ebbe3384808a243440a02c63001" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 37419 } }
I20260809 08:54:44.288653 31817 leader_election.cc:302] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b23f9ebbe3384808a243440a02c63001; no voters:
I20260809 08:54:44.288889 31817 leader_election.cc:288] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 08:54:44.288929 31825 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 08:54:44.289044 31825 raft_consensus.cc:686] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [term 2 LEADER]: Becoming Leader. State: Replica: b23f9ebbe3384808a243440a02c63001, State: Running, Role: LEADER
I20260809 08:54:44.289145 31825 consensus_queue.cc:237] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b23f9ebbe3384808a243440a02c63001" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 37419 } }
I20260809 08:54:44.289201 31817 sys_catalog.cc:564] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:54:44.289449 31831 sys_catalog.cc:455] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b23f9ebbe3384808a243440a02c63001. Latest consensus state: current_term: 2 leader_uuid: "b23f9ebbe3384808a243440a02c63001" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b23f9ebbe3384808a243440a02c63001" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 37419 } } }
I20260809 08:54:44.289714 31831 sys_catalog.cc:458] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [sys.catalog]: This master's current role is: LEADER
I20260809 08:54:44.289636 31830 sys_catalog.cc:455] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b23f9ebbe3384808a243440a02c63001" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b23f9ebbe3384808a243440a02c63001" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 37419 } } }
I20260809 08:54:44.290012 31830 sys_catalog.cc:458] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [sys.catalog]: This master's current role is: LEADER
I20260809 08:54:44.290045 31840 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:54:44.290997 31840 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:54:44.291122 31840 catalog_manager.cc:1259] Loaded cluster ID: 1e6951e211704a35ade02a0bb7a08b54
I20260809 08:54:44.291180 31840 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:54:44.291630 31840 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 08:54:44.294445 31840 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 08:54:44.297087 31840 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:54:44.297338 31840 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001: Loaded TSK: 0
I20260809 08:54:44.297561 31840 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 08:54:44.383747 31819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:54:44.383972 31819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:54:44.383996 31819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:54:44.385349 31819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:54:44.385397 31819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:54:44.385456 31819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.1
I20260809 08:54:44.386947 31819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:36087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.1:43327
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.154.1
--webserver_port=34047
--tserver_master_addrs=127.8.154.62:37419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.1
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:54:44.387230 31819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:54:44.387475 31819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:54:44.392943 31858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:44.393030 31857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:44.393133 31860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:54:44.393359 31819 server_base.cc:1034] running on GCE node
I20260809 08:54:44.393597 31819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:54:44.393877 31819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:54:44.395066 31819 hybrid_clock.cc:648] HybridClock initialized: now 1786265684395046 us; error 35 us; skew 500 ppm
Aug 09 08:54:44 dist-test-slave-t3q3 krb5kdc[30877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265684, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:54:44.397734 31819 init.cc:377] Logged in from keytab as kudu/127.8.154.1@KRBTEST.COM (short username kudu)
I20260809 08:54:44.398219 31819 webserver.cc:460] Webserver started at http://127.8.154.1:34047/ using document root <none> and password file <none>
I20260809 08:54:44.398468 31819 fs_manager.cc:359] Metadata directory not provided
I20260809 08:54:44.398563 31819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:54:44.400002 31819 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:54:44.400780 31869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:44.401029 31819 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:54:44.401162 31819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/wal
uuid: "4715c949a24b43a3a679777781f0c09b"
format_stamp: "Formatted at 2026-08-09 08:54:43 on dist-test-slave-t3q3"
server_key: "d76fc1bc985111a550198fab0c5f36a8"
server_key_iv: "7f3993954bbffad082306b0d14120418"
server_key_version: "encryptionkey@0"
I20260809 08:54:44.401558 31819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:54:44.421136 31819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:54:44.423751 31819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:54:44.423945 31819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:54:44.424269 31819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:54:44.424806 31876 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 08:54:44.426455 31819 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 08:54:44.426517 31819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 08:54:44.426545 31819 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 08:54:44.427266 31819 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 08:54:44.427325 31819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 08:54:44.427363 31876 tablet_bootstrap.cc:492] T 078d3f7bff294fca8751a6d153e4e5aa P 4715c949a24b43a3a679777781f0c09b: Bootstrap starting.
I20260809 08:54:44.435559 31819 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.1:43327
I20260809 08:54:44.435806 32041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.1:43327 every 8 connection(s)
I20260809 08:54:44.436165 31819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
I20260809 08:54:44.439126 31876 log.cc:824] T 078d3f7bff294fca8751a6d153e4e5aa P 4715c949a24b43a3a679777781f0c09b: Log is configured to *not* fsync() on all Append() calls
I20260809 08:54:44.441517 31876 tablet_bootstrap.cc:492] T 078d3f7bff294fca8751a6d153e4e5aa P 4715c949a24b43a3a679777781f0c09b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 08:54:44.441815 31876 tablet_bootstrap.cc:492] T 078d3f7bff294fca8751a6d153e4e5aa P 4715c949a24b43a3a679777781f0c09b: Bootstrap complete.
I20260809 08:54:44.442780 31876 ts_tablet_manager.cc:1397] T 078d3f7bff294fca8751a6d153e4e5aa P 4715c949a24b43a3a679777781f0c09b: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.009s
I20260809 08:54:44.444543 31876 raft_consensus.cc:348] T 078d3f7bff294fca8751a6d153e4e5aa P 4715c949a24b43a3a679777781f0c09b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8561daa539d243618585ad07d97c21cb" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 35211 } } peers { permanent_uuid: "0201a5d9a2104dc397da46d707c50b73" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43885 } } peers { permanent_uuid: "4715c949a24b43a3a679777781f0c09b" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43327 } }
I20260809 08:54:44.444871 31876 raft_consensus.cc:729] T 078d3f7bff294fca8751a6d153e4e5aa P 4715c949a24b43a3a679777781f0c09b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4715c949a24b43a3a679777781f0c09b, State: Initialized, Role: FOLLOWER
I20260809 08:54:44.445004 31876 consensus_queue.cc:260] T 078d3f7bff294fca8751a6d153e4e5aa P 4715c949a24b43a3a679777781f0c09b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8561daa539d243618585ad07d97c21cb" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 35211 } } peers { permanent_uuid: "0201a5d9a2104dc397da46d707c50b73" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43885 } } peers { permanent_uuid: "4715c949a24b43a3a679777781f0c09b" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43327 } }
I20260809 08:54:44.445273 31876 ts_tablet_manager.cc:1428] T 078d3f7bff294fca8751a6d153e4e5aa P 4715c949a24b43a3a679777781f0c09b: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
I20260809 08:54:44.445928 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 31819
I20260809 08:54:44.446182 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.2:43885
--local_ip_for_outbound_sockets=127.8.154.2
--tserver_master_addrs=127.8.154.62:37419
--webserver_port=38757
--webserver_interface=127.8.154.2
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:36087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 08:54:44 dist-test-slave-t3q3 krb5kdc[30877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265684, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:54:44.454829 32049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:44.440000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:49067 (local address 127.8.154.62:37419)
0809 08:54:44.440232 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:44.440241 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:44.443220 (+ 2979us) server_negotiation.cc:407] Connection header received
0809 08:54:44.444342 (+ 1122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:44.444348 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:44.444390 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:44.444496 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:54:44.445362 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.446466 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.447976 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.448628 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.448771 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:44.451227 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:54:44.451231 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:54:44.451232 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:54:44.451244 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:54:44.453060 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:44.453528 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:44.453531 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:44.453532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:44.453627 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:44.453891 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:44.453891 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:44.453892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:44.454110 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:54:44.454208 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:44.454508 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 08:54:44.454633 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":114,"threads_started":1}
I20260809 08:54:44.454829 32047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:44.439909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.1:49067)
0809 08:54:44.443007 (+ 3098us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:44.443053 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 08:54:44.444114 (+ 1061us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:54:44.444567 (+ 453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:54:44.444571 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:54:44.444946 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:54:44.445237 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:44.445243 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.446557 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.446558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:44.447875 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:44.447878 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.448741 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.448742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:44.448847 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:44.449451 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:54:44.449469 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:54:44.451118 (+ 1649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:54:44.453198 (+ 2080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:44.453205 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:44.453207 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:44.453440 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:44.453704 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:44.453705 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:44.453705 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:44.453820 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:44.454201 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:54:44.454208 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:54:44.454346 (+ 138us) client_negotiation.cc:769] Sending connection context
0809 08:54:44.454454 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 08:54:44.454631 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3053,"thread_start_us":72,"threads_started":1}
I20260809 08:54:44.455475 32044 heartbeater.cc:347] Connected to a master server at 127.8.154.62:37419
I20260809 08:54:44.455592 32044 heartbeater.cc:464] Registering TS with master...
I20260809 08:54:44.455806 32044 heartbeater.cc:511] Master 127.8.154.62:37419 requested a full tablet report, sending...
I20260809 08:54:44.456588 31761 ts_manager.cc:194] Registered new tserver with Master: 4715c949a24b43a3a679777781f0c09b (127.8.154.1:43327)
I20260809 08:54:44.457648 31761 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.1@KRBTEST.COM'} at 127.8.154.1:49067
I20260809 08:54:44.458482 32044 heartbeater.cc:503] Master 127.8.154.62:37419 was elected leader, sending a full tablet report...
I20260809 08:54:44.463665 32054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:44.457267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60152 (local address 127.8.154.1:43327)
0809 08:54:44.460368 (+ 3101us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:44.460372 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:44.460379 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:54:44.460403 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:44.460404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:44.460416 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:44.460473 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:54:44.460661 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.461395 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.462850 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.463286 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.463343 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:44.463377 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:44.463589 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 08:54:44.463619 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3075,"thread_start_us":84,"threads_started":1}
I20260809 08:54:44.463730 32053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:44.457185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:43327 (local address 127.0.0.1:60152)
0809 08:54:44.460186 (+ 3001us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:44.460215 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 08:54:44.460266 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:54:44.460507 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:54:44.460511 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:54:44.460513 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:54:44.460583 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:44.460585 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.461468 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.461469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:44.462767 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:44.462770 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.463370 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.463372 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:44.463471 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:44.463487 (+ 16us) client_negotiation.cc:769] Sending connection context
0809 08:54:44.463571 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 08:54:44.463641 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2977,"thread_start_us":93,"threads_started":1}
I20260809 08:54:44.464156 31944 tablet_service.cc:1514] Processing DeleteTablet for tablet 078d3f7bff294fca8751a6d153e4e5aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:54:44 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:60152
I20260809 08:54:44.464725 32055 tablet_replica.cc:330] stopping tablet replica
I20260809 08:54:44.464844 32055 raft_consensus.cc:2229] T 078d3f7bff294fca8751a6d153e4e5aa P 4715c949a24b43a3a679777781f0c09b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:54:44.464927 32055 raft_consensus.cc:2258] T 078d3f7bff294fca8751a6d153e4e5aa P 4715c949a24b43a3a679777781f0c09b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:54:44.465234 32055 ts_tablet_manager.cc:1905] T 078d3f7bff294fca8751a6d153e4e5aa P 4715c949a24b43a3a679777781f0c09b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:54:44.467049 32055 ts_tablet_manager.cc:1918] T 078d3f7bff294fca8751a6d153e4e5aa P 4715c949a24b43a3a679777781f0c09b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:54:44.467155 32055 log.cc:1196] T 078d3f7bff294fca8751a6d153e4e5aa P 4715c949a24b43a3a679777781f0c09b: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/wal/wals/078d3f7bff294fca8751a6d153e4e5aa
I20260809 08:54:44.467396 32055 ts_tablet_manager.cc:1939] T 078d3f7bff294fca8751a6d153e4e5aa P 4715c949a24b43a3a679777781f0c09b: Deleting consensus metadata
I20260809 08:54:44.467876 31738 catalog_manager.cc:4841] TS 4715c949a24b43a3a679777781f0c09b (127.8.154.1:43327): tablet 078d3f7bff294fca8751a6d153e4e5aa (table test-table [id=65842aa01fcc468d9bbf3cc5aa45efc5]) successfully deleted
W20260809 08:54:44.531903 32051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:54:44.532125 32051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:54:44.532151 32051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:54:44.533308 32051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:54:44.533351 32051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:54:44.533403 32051 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.2
I20260809 08:54:44.534847 32051 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:36087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.2:43885
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.154.2
--webserver_port=38757
--tserver_master_addrs=127.8.154.62:37419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.2
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:54:44.535084 32051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:54:44.535323 32051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:54:44.540499 32068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:44.540522 32063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:54:44.540545 32051 server_base.cc:1034] running on GCE node
W20260809 08:54:44.540493 32062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:54:44.540895 32051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:54:44.541155 32051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:54:44.542321 32051 hybrid_clock.cc:648] HybridClock initialized: now 1786265684542309 us; error 44 us; skew 500 ppm
Aug 09 08:54:44 dist-test-slave-t3q3 krb5kdc[30877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265684, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:54:44.544799 32051 init.cc:377] Logged in from keytab as kudu/127.8.154.2@KRBTEST.COM (short username kudu)
I20260809 08:54:44.545264 32051 webserver.cc:460] Webserver started at http://127.8.154.2:38757/ using document root <none> and password file <none>
I20260809 08:54:44.545506 32051 fs_manager.cc:359] Metadata directory not provided
I20260809 08:54:44.545588 32051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:54:44.546967 32051 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:54:44.547614 32077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:44.547803 32051 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:54:44.547901 32051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/wal
uuid: "0201a5d9a2104dc397da46d707c50b73"
format_stamp: "Formatted at 2026-08-09 08:54:43 on dist-test-slave-t3q3"
server_key: "1ffd114d8b75f47462163f7358c26adf"
server_key_iv: "11db7ede9c150dc23c774fa4925a19d0"
server_key_version: "encryptionkey@0"
I20260809 08:54:44.548295 32051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:54:44.570353 32051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:54:44.572664 32051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:54:44.572783 32051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:54:44.572956 32051 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:54:44.573318 32051 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:54:44.573354 32051 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:44.573376 32051 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:54:44.573423 32051 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:44.579295 32051 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.2:43885
I20260809 08:54:44.579376 32242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.2:43885 every 8 connection(s)
I20260809 08:54:44.579710 32051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
I20260809 08:54:44.582736 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 32051
I20260809 08:54:44.582989 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.3:35211
--local_ip_for_outbound_sockets=127.8.154.3
--tserver_master_addrs=127.8.154.62:37419
--webserver_port=46567
--webserver_interface=127.8.154.3
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:36087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 08:54:44 dist-test-slave-t3q3 krb5kdc[30877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265684, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:54:44.590442 32049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:44.580488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:44245 (local address 127.8.154.62:37419)
0809 08:54:44.580534 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:44.580536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:44.580738 (+ 202us) server_negotiation.cc:407] Connection header received
0809 08:54:44.581288 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:44.581290 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:44.581307 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:44.581371 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:54:44.582325 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.583081 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.584193 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.584538 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.584620 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:44.587129 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:54:44.587130 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:54:44.587130 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:54:44.587137 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:54:44.588654 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:44.589223 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:44.589224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:44.589225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:44.589291 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:44.589606 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:44.589607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:44.589607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:44.589769 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:54:44.589868 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:44.590289 (+ 421us) server_negotiation.cc:299] Negotiation successful
0809 08:54:44.590378 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 08:54:44.590643 32245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:44.580454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.2:44245)
0809 08:54:44.580608 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:44.580646 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 08:54:44.581175 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:54:44.581446 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:54:44.581450 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:54:44.582024 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:54:44.582220 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:44.582225 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.583205 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.583206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:44.584051 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:44.584053 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.584642 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.584643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:44.584811 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:44.585313 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:54:44.585323 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:54:44.587003 (+ 1680us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:54:44.588731 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:44.588738 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:44.588740 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:44.589083 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:44.589379 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:44.589380 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:44.589381 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:44.589532 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:44.589860 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:54:44.589867 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:54:44.590055 (+ 188us) client_negotiation.cc:769] Sending connection context
0809 08:54:44.590186 (+ 131us) client_negotiation.cc:240] Negotiation successful
0809 08:54:44.590370 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":71,"threads_started":1}
I20260809 08:54:44.591138 32243 heartbeater.cc:347] Connected to a master server at 127.8.154.62:37419
I20260809 08:54:44.591259 32243 heartbeater.cc:464] Registering TS with master...
I20260809 08:54:44.591434 32243 heartbeater.cc:511] Master 127.8.154.62:37419 requested a full tablet report, sending...
I20260809 08:54:44.591862 31761 ts_manager.cc:194] Registered new tserver with Master: 0201a5d9a2104dc397da46d707c50b73 (127.8.154.2:43885)
I20260809 08:54:44.592324 31761 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.2@KRBTEST.COM'} at 127.8.154.2:44245
W20260809 08:54:44.667467 32246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:54:44.667694 32246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:54:44.667732 32246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:54:44.668877 32246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:54:44.668942 32246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:54:44.668988 32246 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.3
I20260809 08:54:44.670468 32246 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:36087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.3:35211
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.154.3
--webserver_port=46567
--tserver_master_addrs=127.8.154.62:37419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.3
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:54:44.670728 32246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:54:44.670971 32246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:54:44.676090 32258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:44.676083 32261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:44.676107 32265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:54:44.676280 32246 server_base.cc:1034] running on GCE node
I20260809 08:54:44.676539 32246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:54:44.676793 32246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:54:44.677953 32246 hybrid_clock.cc:648] HybridClock initialized: now 1786265684677946 us; error 35 us; skew 500 ppm
Aug 09 08:54:44 dist-test-slave-t3q3 krb5kdc[30877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265684, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:54:44.680629 32246 init.cc:377] Logged in from keytab as kudu/127.8.154.3@KRBTEST.COM (short username kudu)
I20260809 08:54:44.681066 32246 webserver.cc:460] Webserver started at http://127.8.154.3:46567/ using document root <none> and password file <none>
I20260809 08:54:44.681288 32246 fs_manager.cc:359] Metadata directory not provided
I20260809 08:54:44.681346 32246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:54:44.682739 32246 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:54:44.683302 32276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:44.683480 32246 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:44.683588 32246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/wal
uuid: "8561daa539d243618585ad07d97c21cb"
format_stamp: "Formatted at 2026-08-09 08:54:43 on dist-test-slave-t3q3"
server_key: "0d8a25b2dda597ccbd3c810d13cce2d2"
server_key_iv: "23abf76242ce79a756dde566f53d4e9b"
server_key_version: "encryptionkey@0"
I20260809 08:54:44.683933 32246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:54:44.700418 32246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:54:44.702795 32246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:54:44.702970 32246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:54:44.703236 32246 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:54:44.703552 32246 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:54:44.703608 32246 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:44.703648 32246 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:54:44.703678 32246 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:44.709906 32246 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.3:35211
I20260809 08:54:44.710032 32455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.3:35211 every 8 connection(s)
I20260809 08:54:44.710359 32246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
Aug 09 08:54:44 dist-test-slave-t3q3 krb5kdc[30877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265684, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:54:44.719448 32049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:44.711184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:38241 (local address 127.8.154.62:37419)
0809 08:54:44.711255 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:44.711256 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:44.711469 (+ 213us) server_negotiation.cc:407] Connection header received
0809 08:54:44.711948 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:44.711949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:44.711960 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:44.712020 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:54:44.712981 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.713495 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.714478 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.714667 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.714725 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:44.716987 (+ 2262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:54:44.716988 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:54:44.716989 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:54:44.716994 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:54:44.718233 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:44.718607 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:44.718608 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:44.718608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:44.718660 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:44.718868 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:44.718868 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:44.718869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:44.718966 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:54:44.719017 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:44.719319 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 08:54:44.719383 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 08:54:44.719702 32458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:44.711161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.3:38241)
0809 08:54:44.711336 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:44.711372 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 08:54:44.711837 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:54:44.712119 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:54:44.712122 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:54:44.712694 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:54:44.712893 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:44.712897 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.713577 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.713578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:44.714397 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:44.714399 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.714745 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.714746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:44.714848 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:44.715417 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:54:44.715425 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:54:44.716891 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:54:44.718309 (+ 1418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:44.718315 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:44.718317 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:44.718534 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:44.718718 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:44.718719 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:44.718719 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:44.718814 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:44.719046 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:54:44.719053 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:54:44.719195 (+ 142us) client_negotiation.cc:769] Sending connection context
0809 08:54:44.719294 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 08:54:44.719422 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":109,"threads_started":1}
I20260809 08:54:44.720055 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 32246
I20260809 08:54:44.720316 32456 heartbeater.cc:347] Connected to a master server at 127.8.154.62:37419
I20260809 08:54:44.720438 32456 heartbeater.cc:464] Registering TS with master...
I20260809 08:54:44.720649 32456 heartbeater.cc:511] Master 127.8.154.62:37419 requested a full tablet report, sending...
I20260809 08:54:44.721104 31761 ts_manager.cc:194] Registered new tserver with Master: 8561daa539d243618585ad07d97c21cb (127.8.154.3:35211)
I20260809 08:54:44.721505 31761 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.3@KRBTEST.COM'} at 127.8.154.3:38241
I20260809 08:54:44.726843 32049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:44.720534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52496 (local address 127.8.154.62:37419)
0809 08:54:44.720579 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:44.720582 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:44.720806 (+ 224us) server_negotiation.cc:407] Connection header received
0809 08:54:44.720906 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:44.720907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:44.720924 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:44.720992 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:54:44.721657 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.722128 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.722887 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.723102 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.723179 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:44.724408 (+ 1229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:54:44.724410 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:54:44.724410 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:54:44.724417 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:54:44.725660 (+ 1243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:44.726018 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:44.726019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:44.726020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:44.726085 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:44.726303 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:44.726304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:44.726304 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:44.726420 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:54:44.726616 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:44.726655 (+ 39us) server_negotiation.cc:299] Negotiation successful
0809 08:54:44.726736 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 08:54:44.727635 8808 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:54:44.731623 32049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:44.727989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52512 (local address 127.8.154.62:37419)
0809 08:54:44.728027 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:44.728028 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:44.728166 (+ 138us) server_negotiation.cc:407] Connection header received
0809 08:54:44.728228 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:44.728228 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:44.728259 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:44.728356 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:54:44.728498 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.729286 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.730603 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.730930 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.731004 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:44.731219 (+ 215us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:54:44.731366 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:54:44.731461 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:44.731516 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 08:54:44.731538 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 08:54:44.732033 31761 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:52512:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:54:44.734517 31761 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:54:44.737653 31944 tablet_service.cc:1467] Processing CreateTablet for tablet 2dbe195ea64c479fb1978c24bcf867f3 (DEFAULT_TABLE table=test-table [id=cc4af061ee274122ac205bce44c010ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:54:44.737864 31944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dbe195ea64c479fb1978c24bcf867f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:54:44.739360 31876 tablet_bootstrap.cc:492] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b: Bootstrap starting.
I20260809 08:54:44.739694 31876 tablet_bootstrap.cc:654] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b: Neither blocks nor log segments found. Creating new log.
I20260809 08:54:44.740340 31876 tablet_bootstrap.cc:492] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b: No bootstrap required, opened a new log
I20260809 08:54:44.740408 31876 ts_tablet_manager.cc:1397] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260809 08:54:44.740523 31876 raft_consensus.cc:348] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8561daa539d243618585ad07d97c21cb" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 35211 } } peers { permanent_uuid: "0201a5d9a2104dc397da46d707c50b73" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43885 } } peers { permanent_uuid: "4715c949a24b43a3a679777781f0c09b" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43327 } }
I20260809 08:54:44.740612 31876 raft_consensus.cc:374] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:54:44.740666 31876 raft_consensus.cc:729] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4715c949a24b43a3a679777781f0c09b, State: Initialized, Role: FOLLOWER
I20260809 08:54:44.740715 31876 consensus_queue.cc:260] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8561daa539d243618585ad07d97c21cb" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 35211 } } peers { permanent_uuid: "0201a5d9a2104dc397da46d707c50b73" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43885 } } peers { permanent_uuid: "4715c949a24b43a3a679777781f0c09b" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43327 } }
I20260809 08:54:44.740818 31876 ts_tablet_manager.cc:1428] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:44.742240 32463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:44.737377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34746 (local address 127.8.154.3:35211)
0809 08:54:44.737696 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:44.737700 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:44.737711 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:54:44.737756 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:44.737759 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:44.737774 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:44.738380 (+ 606us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:54:44.738469 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.739714 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.741135 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.741918 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.741977 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:44.742022 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:44.742141 (+ 119us) server_negotiation.cc:299] Negotiation successful
0809 08:54:44.742171 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":109,"threads_started":1}
I20260809 08:54:44.742261 32053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:44.737163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:35211 (local address 127.0.0.1:34746)
0809 08:54:44.737234 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:44.737248 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 08:54:44.737321 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:54:44.738232 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:54:44.738233 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:54:44.738235 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:54:44.738287 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:44.738289 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.739897 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.739898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:44.741048 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:44.741051 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.741998 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.741999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:44.742061 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:44.742064 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:54:44.742131 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 08:54:44.742179 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23,"mutex_wait_us":80}
I20260809 08:54:44.742463 32464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:44.737505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56670 (local address 127.8.154.2:43885)
0809 08:54:44.737694 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:44.737700 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:44.738331 (+ 631us) server_negotiation.cc:407] Connection header received
0809 08:54:44.738407 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:44.738410 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:44.738426 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:44.738509 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:54:44.738649 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.739748 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.741257 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.741924 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.742014 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:44.742075 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:44.742306 (+ 231us) server_negotiation.cc:299] Negotiation successful
0809 08:54:44.742357 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":77,"threads_started":1}
I20260809 08:54:44.742705 32462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:44.737195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:43885 (local address 127.0.0.1:56670)
0809 08:54:44.738049 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:44.738060 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:54:44.738124 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:54:44.738521 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:54:44.738522 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:54:44.738524 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:54:44.738572 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:44.738573 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.739859 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.739860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:44.741194 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:44.741197 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.742119 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.742120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:44.742174 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:44.742177 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:54:44.742223 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 08:54:44.742257 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":838,"mutex_wait_us":36,"thread_start_us":66,"threads_started":1}
I20260809 08:54:44.743098 32351 tablet_service.cc:1467] Processing CreateTablet for tablet 2dbe195ea64c479fb1978c24bcf867f3 (DEFAULT_TABLE table=test-table [id=cc4af061ee274122ac205bce44c010ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:54:44.743484 32351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dbe195ea64c479fb1978c24bcf867f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:54:44.743438 32142 tablet_service.cc:1467] Processing CreateTablet for tablet 2dbe195ea64c479fb1978c24bcf867f3 (DEFAULT_TABLE table=test-table [id=cc4af061ee274122ac205bce44c010ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:54:44.743749 32142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2dbe195ea64c479fb1978c24bcf867f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:54:44.746297 32467 tablet_bootstrap.cc:492] T 2dbe195ea64c479fb1978c24bcf867f3 P 8561daa539d243618585ad07d97c21cb: Bootstrap starting.
I20260809 08:54:44.746593 32469 tablet_bootstrap.cc:492] T 2dbe195ea64c479fb1978c24bcf867f3 P 0201a5d9a2104dc397da46d707c50b73: Bootstrap starting.
I20260809 08:54:44.747031 32467 tablet_bootstrap.cc:654] T 2dbe195ea64c479fb1978c24bcf867f3 P 8561daa539d243618585ad07d97c21cb: Neither blocks nor log segments found. Creating new log.
I20260809 08:54:44.747251 32469 tablet_bootstrap.cc:654] T 2dbe195ea64c479fb1978c24bcf867f3 P 0201a5d9a2104dc397da46d707c50b73: Neither blocks nor log segments found. Creating new log.
I20260809 08:54:44.747738 32467 log.cc:824] T 2dbe195ea64c479fb1978c24bcf867f3 P 8561daa539d243618585ad07d97c21cb: Log is configured to *not* fsync() on all Append() calls
I20260809 08:54:44.747857 32469 log.cc:824] T 2dbe195ea64c479fb1978c24bcf867f3 P 0201a5d9a2104dc397da46d707c50b73: Log is configured to *not* fsync() on all Append() calls
I20260809 08:54:44.748608 32467 tablet_bootstrap.cc:492] T 2dbe195ea64c479fb1978c24bcf867f3 P 8561daa539d243618585ad07d97c21cb: No bootstrap required, opened a new log
I20260809 08:54:44.748644 32469 tablet_bootstrap.cc:492] T 2dbe195ea64c479fb1978c24bcf867f3 P 0201a5d9a2104dc397da46d707c50b73: No bootstrap required, opened a new log
I20260809 08:54:44.748699 32467 ts_tablet_manager.cc:1397] T 2dbe195ea64c479fb1978c24bcf867f3 P 8561daa539d243618585ad07d97c21cb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:54:44.748718 32469 ts_tablet_manager.cc:1397] T 2dbe195ea64c479fb1978c24bcf867f3 P 0201a5d9a2104dc397da46d707c50b73: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 08:54:44.750026 32467 raft_consensus.cc:348] T 2dbe195ea64c479fb1978c24bcf867f3 P 8561daa539d243618585ad07d97c21cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8561daa539d243618585ad07d97c21cb" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 35211 } } peers { permanent_uuid: "0201a5d9a2104dc397da46d707c50b73" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43885 } } peers { permanent_uuid: "4715c949a24b43a3a679777781f0c09b" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43327 } }
I20260809 08:54:44.750201 32467 raft_consensus.cc:374] T 2dbe195ea64c479fb1978c24bcf867f3 P 8561daa539d243618585ad07d97c21cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:54:44.750253 32467 raft_consensus.cc:729] T 2dbe195ea64c479fb1978c24bcf867f3 P 8561daa539d243618585ad07d97c21cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8561daa539d243618585ad07d97c21cb, State: Initialized, Role: FOLLOWER
I20260809 08:54:44.750252 32469 raft_consensus.cc:348] T 2dbe195ea64c479fb1978c24bcf867f3 P 0201a5d9a2104dc397da46d707c50b73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8561daa539d243618585ad07d97c21cb" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 35211 } } peers { permanent_uuid: "0201a5d9a2104dc397da46d707c50b73" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43885 } } peers { permanent_uuid: "4715c949a24b43a3a679777781f0c09b" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43327 } }
I20260809 08:54:44.750422 32467 consensus_queue.cc:260] T 2dbe195ea64c479fb1978c24bcf867f3 P 8561daa539d243618585ad07d97c21cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8561daa539d243618585ad07d97c21cb" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 35211 } } peers { permanent_uuid: "0201a5d9a2104dc397da46d707c50b73" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43885 } } peers { permanent_uuid: "4715c949a24b43a3a679777781f0c09b" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43327 } }
I20260809 08:54:44.750458 32469 raft_consensus.cc:374] T 2dbe195ea64c479fb1978c24bcf867f3 P 0201a5d9a2104dc397da46d707c50b73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:54:44.750512 32469 raft_consensus.cc:729] T 2dbe195ea64c479fb1978c24bcf867f3 P 0201a5d9a2104dc397da46d707c50b73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0201a5d9a2104dc397da46d707c50b73, State: Initialized, Role: FOLLOWER
I20260809 08:54:44.750628 32469 consensus_queue.cc:260] T 2dbe195ea64c479fb1978c24bcf867f3 P 0201a5d9a2104dc397da46d707c50b73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8561daa539d243618585ad07d97c21cb" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 35211 } } peers { permanent_uuid: "0201a5d9a2104dc397da46d707c50b73" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43885 } } peers { permanent_uuid: "4715c949a24b43a3a679777781f0c09b" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43327 } }
I20260809 08:54:44.750748 32467 ts_tablet_manager.cc:1428] T 2dbe195ea64c479fb1978c24bcf867f3 P 8561daa539d243618585ad07d97c21cb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:54:44.750873 32456 heartbeater.cc:503] Master 127.8.154.62:37419 was elected leader, sending a full tablet report...
I20260809 08:54:44.750924 32469 ts_tablet_manager.cc:1428] T 2dbe195ea64c479fb1978c24bcf867f3 P 0201a5d9a2104dc397da46d707c50b73: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 08:54:44.751423 32243 heartbeater.cc:503] Master 127.8.154.62:37419 was elected leader, sending a full tablet report...
I20260809 08:54:44.759691 32472 raft_consensus.cc:482] T 2dbe195ea64c479fb1978c24bcf867f3 P 8561daa539d243618585ad07d97c21cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:54:44.759779 32472 raft_consensus.cc:504] T 2dbe195ea64c479fb1978c24bcf867f3 P 8561daa539d243618585ad07d97c21cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8561daa539d243618585ad07d97c21cb" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 35211 } } peers { permanent_uuid: "0201a5d9a2104dc397da46d707c50b73" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43885 } } peers { permanent_uuid: "4715c949a24b43a3a679777781f0c09b" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43327 } }
I20260809 08:54:44.760104 32472 leader_election.cc:288] T 2dbe195ea64c479fb1978c24bcf867f3 P 8561daa539d243618585ad07d97c21cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0201a5d9a2104dc397da46d707c50b73 (127.8.154.2:43885), 4715c949a24b43a3a679777781f0c09b (127.8.154.1:43327)
I20260809 08:54:44.762929 32458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:44.760239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:43885 (local address 127.8.154.3:35813)
0809 08:54:44.760280 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:44.760295 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 08:54:44.760345 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:54:44.760481 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:54:44.760482 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:54:44.760483 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:54:44.760548 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:44.760549 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.761214 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.761215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:44.762023 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:44.762027 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.762653 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.762655 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:44.762742 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:44.762745 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:54:44.762821 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 08:54:44.762863 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260809 08:54:44.762929 32464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:44.760321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:35813 (local address 127.8.154.2:43885)
0809 08:54:44.760376 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:44.760378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:44.760386 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:54:44.760411 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:44.760411 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:44.760421 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:44.760477 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:54:44.760637 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.761127 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.762100 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.762549 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.762651 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:44.762663 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:44.762830 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 08:54:44.762864 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 08:54:44.763218 32475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:44.760239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:43327 (local address 127.8.154.3:53149)
0809 08:54:44.760471 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:44.760480 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:54:44.760568 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:54:44.760721 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:54:44.760722 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:54:44.760723 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:54:44.760791 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:44.760793 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.761429 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.761430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:44.762368 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:44.762371 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.762917 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.762918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:44.762993 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:44.762998 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:54:44.763091 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 08:54:44.763135 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":80,"thread_start_us":102,"threads_started":1}
I20260809 08:54:44.763288 32054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:44.760342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:53149 (local address 127.8.154.1:43327)
0809 08:54:44.760390 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:44.760393 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:44.760554 (+ 161us) server_negotiation.cc:407] Connection header received
0809 08:54:44.760647 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:44.760648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:44.760661 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:44.760718 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:54:44.760877 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.761342 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.762455 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.762826 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.762915 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:44.762925 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:44.763168 (+ 243us) server_negotiation.cc:299] Negotiation successful
0809 08:54:44.763221 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 08:54:44.763269 32174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dbe195ea64c479fb1978c24bcf867f3" candidate_uuid: "8561daa539d243618585ad07d97c21cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0201a5d9a2104dc397da46d707c50b73" is_pre_election: true
I20260809 08:54:44.763406 32174 raft_consensus.cc:2454] T 2dbe195ea64c479fb1978c24bcf867f3 P 0201a5d9a2104dc397da46d707c50b73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8561daa539d243618585ad07d97c21cb in term 0.
I20260809 08:54:44.763669 31970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dbe195ea64c479fb1978c24bcf867f3" candidate_uuid: "8561daa539d243618585ad07d97c21cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4715c949a24b43a3a679777781f0c09b" is_pre_election: true
I20260809 08:54:44.763819 31970 raft_consensus.cc:2454] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8561daa539d243618585ad07d97c21cb in term 0.
I20260809 08:54:44.763999 32278 leader_election.cc:302] T 2dbe195ea64c479fb1978c24bcf867f3 P 8561daa539d243618585ad07d97c21cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0201a5d9a2104dc397da46d707c50b73, 8561daa539d243618585ad07d97c21cb; no voters:
I20260809 08:54:44.764182 32472 raft_consensus.cc:2781] T 2dbe195ea64c479fb1978c24bcf867f3 P 8561daa539d243618585ad07d97c21cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:54:44.764249 32472 raft_consensus.cc:482] T 2dbe195ea64c479fb1978c24bcf867f3 P 8561daa539d243618585ad07d97c21cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:54:44.764285 32472 raft_consensus.cc:3037] T 2dbe195ea64c479fb1978c24bcf867f3 P 8561daa539d243618585ad07d97c21cb [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:54:44.764973 32472 raft_consensus.cc:504] T 2dbe195ea64c479fb1978c24bcf867f3 P 8561daa539d243618585ad07d97c21cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8561daa539d243618585ad07d97c21cb" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 35211 } } peers { permanent_uuid: "0201a5d9a2104dc397da46d707c50b73" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43885 } } peers { permanent_uuid: "4715c949a24b43a3a679777781f0c09b" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43327 } }
I20260809 08:54:44.765123 32472 leader_election.cc:288] T 2dbe195ea64c479fb1978c24bcf867f3 P 8561daa539d243618585ad07d97c21cb [CANDIDATE]: Term 1 election: Requested vote from peers 0201a5d9a2104dc397da46d707c50b73 (127.8.154.2:43885), 4715c949a24b43a3a679777781f0c09b (127.8.154.1:43327)
I20260809 08:54:44.765264 31970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dbe195ea64c479fb1978c24bcf867f3" candidate_uuid: "8561daa539d243618585ad07d97c21cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4715c949a24b43a3a679777781f0c09b"
I20260809 08:54:44.765281 32174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2dbe195ea64c479fb1978c24bcf867f3" candidate_uuid: "8561daa539d243618585ad07d97c21cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0201a5d9a2104dc397da46d707c50b73"
I20260809 08:54:44.765340 31970 raft_consensus.cc:3037] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:54:44.765383 32174 raft_consensus.cc:3037] T 2dbe195ea64c479fb1978c24bcf867f3 P 0201a5d9a2104dc397da46d707c50b73 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:54:44.766088 32174 raft_consensus.cc:2454] T 2dbe195ea64c479fb1978c24bcf867f3 P 0201a5d9a2104dc397da46d707c50b73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8561daa539d243618585ad07d97c21cb in term 1.
I20260809 08:54:44.766297 32278 leader_election.cc:302] T 2dbe195ea64c479fb1978c24bcf867f3 P 8561daa539d243618585ad07d97c21cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0201a5d9a2104dc397da46d707c50b73, 8561daa539d243618585ad07d97c21cb; no voters:
I20260809 08:54:44.766389 32472 raft_consensus.cc:2781] T 2dbe195ea64c479fb1978c24bcf867f3 P 8561daa539d243618585ad07d97c21cb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:54:44.766551 32472 raft_consensus.cc:686] T 2dbe195ea64c479fb1978c24bcf867f3 P 8561daa539d243618585ad07d97c21cb [term 1 LEADER]: Becoming Leader. State: Replica: 8561daa539d243618585ad07d97c21cb, State: Running, Role: LEADER
I20260809 08:54:44.766578 31970 raft_consensus.cc:2454] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8561daa539d243618585ad07d97c21cb in term 1.
I20260809 08:54:44.766656 32472 consensus_queue.cc:237] T 2dbe195ea64c479fb1978c24bcf867f3 P 8561daa539d243618585ad07d97c21cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8561daa539d243618585ad07d97c21cb" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 35211 } } peers { permanent_uuid: "0201a5d9a2104dc397da46d707c50b73" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43885 } } peers { permanent_uuid: "4715c949a24b43a3a679777781f0c09b" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43327 } }
I20260809 08:54:44.767458 31760 catalog_manager.cc:5495] T 2dbe195ea64c479fb1978c24bcf867f3 P 8561daa539d243618585ad07d97c21cb reported cstate change: term changed from 0 to 1, leader changed from <none> to 8561daa539d243618585ad07d97c21cb (127.8.154.3). New cstate: current_term: 1 leader_uuid: "8561daa539d243618585ad07d97c21cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8561daa539d243618585ad07d97c21cb" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 35211 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0201a5d9a2104dc397da46d707c50b73" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4715c949a24b43a3a679777781f0c09b" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43327 } health_report { overall_health: UNKNOWN } } }
I20260809 08:54:44.772529 32463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:44.770197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34752 (local address 127.8.154.3:35211)
0809 08:54:44.770263 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:44.770265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:44.770272 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:54:44.770307 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:44.770308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:44.770325 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:44.770397 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:54:44.770520 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.770975 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.771706 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:44.771916 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:44.771992 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:44.772193 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:54:44.772301 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:54:44.772394 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:44.772414 (+ 20us) server_negotiation.cc:299] Negotiation successful
0809 08:54:44.772434 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 08:54:44.774222 31970 raft_consensus.cc:1261] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [term 1 FOLLOWER]: Refusing update from remote peer 8561daa539d243618585ad07d97c21cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:54:44.774225 32174 raft_consensus.cc:1261] T 2dbe195ea64c479fb1978c24bcf867f3 P 0201a5d9a2104dc397da46d707c50b73 [term 1 FOLLOWER]: Refusing update from remote peer 8561daa539d243618585ad07d97c21cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:54:44.774562 32478 consensus_queue.cc:1035] T 2dbe195ea64c479fb1978c24bcf867f3 P 8561daa539d243618585ad07d97c21cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "4715c949a24b43a3a679777781f0c09b" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:54:44.774717 32472 consensus_queue.cc:1035] T 2dbe195ea64c479fb1978c24bcf867f3 P 8561daa539d243618585ad07d97c21cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "0201a5d9a2104dc397da46d707c50b73" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:54:44.777019 32481 mvcc.cc:205] Tried to move back new op lower bound from 7316544244832788480 to 7316544244805062656. Current Snapshot: MvccSnapshot[applied={T|T < 7316544244832788480}]
I20260809 08:54:44.777472 32482 mvcc.cc:205] Tried to move back new op lower bound from 7316544244832788480 to 7316544244805062656. Current Snapshot: MvccSnapshot[applied={T|T < 7316544244832788480 or (T in {7316544244832788480})}]
I20260809 08:54:44.780141 31757 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:52512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:54:44.780247 31757 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:52512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:54:44.780970 31757 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001: Sending DeleteTablet for 3 replicas of tablet 2dbe195ea64c479fb1978c24bcf867f3
I20260809 08:54:44.781347 31944 tablet_service.cc:1514] Processing DeleteTablet for tablet 2dbe195ea64c479fb1978c24bcf867f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:54:44 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:60152
I20260809 08:54:44.781344 32351 tablet_service.cc:1514] Processing DeleteTablet for tablet 2dbe195ea64c479fb1978c24bcf867f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:54:44 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:34746
I20260809 08:54:44.781397 32142 tablet_service.cc:1514] Processing DeleteTablet for tablet 2dbe195ea64c479fb1978c24bcf867f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:54:44 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:56670
I20260809 08:54:44.781448 32055 tablet_replica.cc:330] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b: stopping tablet replica
I20260809 08:54:44.781497 32055 raft_consensus.cc:2229] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:54:44.781543 32055 raft_consensus.cc:2258] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:54:44.781629 32494 tablet_replica.cc:330] T 2dbe195ea64c479fb1978c24bcf867f3 P 0201a5d9a2104dc397da46d707c50b73: stopping tablet replica
I20260809 08:54:44.781679 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 31819
I20260809 08:54:44.781733 32494 raft_consensus.cc:2229] T 2dbe195ea64c479fb1978c24bcf867f3 P 0201a5d9a2104dc397da46d707c50b73 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:54:44.781831 32494 raft_consensus.cc:2258] T 2dbe195ea64c479fb1978c24bcf867f3 P 0201a5d9a2104dc397da46d707c50b73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:54:44.782302 32494 ts_tablet_manager.cc:1905] T 2dbe195ea64c479fb1978c24bcf867f3 P 0201a5d9a2104dc397da46d707c50b73: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:54:44.784109 32494 ts_tablet_manager.cc:1918] T 2dbe195ea64c479fb1978c24bcf867f3 P 0201a5d9a2104dc397da46d707c50b73: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:54:44.784186 32494 log.cc:1196] T 2dbe195ea64c479fb1978c24bcf867f3 P 0201a5d9a2104dc397da46d707c50b73: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/wal/wals/2dbe195ea64c479fb1978c24bcf867f3
I20260809 08:54:44.784497 32494 ts_tablet_manager.cc:1939] T 2dbe195ea64c479fb1978c24bcf867f3 P 0201a5d9a2104dc397da46d707c50b73: Deleting consensus metadata
I20260809 08:54:44.784909 31734 catalog_manager.cc:4841] TS 0201a5d9a2104dc397da46d707c50b73 (127.8.154.2:43885): tablet 2dbe195ea64c479fb1978c24bcf867f3 (table test-table [id=cc4af061ee274122ac205bce44c010ae]) successfully deleted
I20260809 08:54:44.791556 32493 tablet_replica.cc:330] T 2dbe195ea64c479fb1978c24bcf867f3 P 8561daa539d243618585ad07d97c21cb: stopping tablet replica
I20260809 08:54:44.791746 32493 raft_consensus.cc:2229] T 2dbe195ea64c479fb1978c24bcf867f3 P 8561daa539d243618585ad07d97c21cb [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:54:44.791936 32493 raft_consensus.cc:2258] T 2dbe195ea64c479fb1978c24bcf867f3 P 8561daa539d243618585ad07d97c21cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:54:44.792924 32493 ts_tablet_manager.cc:1905] T 2dbe195ea64c479fb1978c24bcf867f3 P 8561daa539d243618585ad07d97c21cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:54:44.794304 32493 ts_tablet_manager.cc:1918] T 2dbe195ea64c479fb1978c24bcf867f3 P 8561daa539d243618585ad07d97c21cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:54:44.794404 32493 log.cc:1196] T 2dbe195ea64c479fb1978c24bcf867f3 P 8561daa539d243618585ad07d97c21cb: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/wal/wals/2dbe195ea64c479fb1978c24bcf867f3
I20260809 08:54:44.794693 32493 ts_tablet_manager.cc:1939] T 2dbe195ea64c479fb1978c24bcf867f3 P 8561daa539d243618585ad07d97c21cb: Deleting consensus metadata
I20260809 08:54:44.795127 31738 catalog_manager.cc:4841] TS 8561daa539d243618585ad07d97c21cb (127.8.154.3:35211): tablet 2dbe195ea64c479fb1978c24bcf867f3 (table test-table [id=cc4af061ee274122ac205bce44c010ae]) successfully deleted
W20260809 08:54:44.798547 31738 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.154.1:43327 (error 108)
I20260809 08:54:44.798954 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 32051
I20260809 08:54:44.799098 32462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:44.798939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:43327 (local address 127.0.0.1:60160)
0809 08:54:44.799011 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:44.799061 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.1:43327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 08:54:44.801226 31738 catalog_manager.cc:4584] TS 4715c949a24b43a3a679777781f0c09b (127.8.154.1:43327): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2dbe195ea64c479fb1978c24bcf867f3: Network error: Client connection negotiation failed: client connection to 127.8.154.1:43327: connect: Connection refused (error 111)
I20260809 08:54:44.805974 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 32246
I20260809 08:54:44.816738 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 31712
I20260809 08:54:44.830168 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:37419
--webserver_interface=127.8.154.62
--webserver_port=36443
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:36087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:37419
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:54:44.914433 32497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:54:44.914633 32497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:54:44.914656 32497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:54:44.915781 32497 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:54:44.915824 32497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:54:44.915838 32497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:54:44.915851 32497 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:54:44.915864 32497 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:54:44.917383 32497 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:36087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:37419
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:37419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=36443
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:54:44.917642 32497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:54:44.917872 32497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:54:44.923096 32508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:44.923067 32511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:44.923076 32505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:54:44.923283 32497 server_base.cc:1034] running on GCE node
I20260809 08:54:44.923511 32497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:54:44.923794 32497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:54:44.924934 32497 hybrid_clock.cc:648] HybridClock initialized: now 1786265684924923 us; error 29 us; skew 500 ppm
Aug 09 08:54:44 dist-test-slave-t3q3 krb5kdc[30877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265684, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:54:44.927356 32497 init.cc:377] Logged in from keytab as kudu/127.8.154.62@KRBTEST.COM (short username kudu)
I20260809 08:54:44.927722 32497 webserver.cc:460] Webserver started at http://127.8.154.62:36443/ using document root <none> and password file <none>
I20260809 08:54:44.927918 32497 fs_manager.cc:359] Metadata directory not provided
I20260809 08:54:44.927959 32497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:54:44.929275 32497 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:54:44.929888 32520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:44.930099 32497 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:54:44.930193 32497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "b23f9ebbe3384808a243440a02c63001"
format_stamp: "Formatted at 2026-08-09 08:54:43 on dist-test-slave-t3q3"
server_key: "d312bb31f3d106be93c32960feaa9dfd"
server_key_iv: "1c91a7ab1a593286f68de47d151d6e0b"
server_key_version: "encryptionkey@0"
I20260809 08:54:44.930527 32497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:54:44.963384 32497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:54:44.965454 32497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:54:44.965626 32497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:54:44.969734 32497 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:37419
I20260809 08:54:44.969800 32598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:37419 every 8 connection(s)
I20260809 08:54:44.970214 32497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:54:44.971860 32599 sys_catalog.cc:263] Verifying existing consensus state
I20260809 08:54:44.972584 32599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001: Bootstrap starting.
I20260809 08:54:44.975956 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 32497
I20260809 08:54:44.976291 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.1:43327
--local_ip_for_outbound_sockets=127.8.154.1
--tserver_master_addrs=127.8.154.62:37419
--webserver_port=34047
--webserver_interface=127.8.154.1
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:36087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:54:44.981360 32599 log.cc:824] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001: Log is configured to *not* fsync() on all Append() calls
I20260809 08:54:44.983631 32599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 08:54:44.983939 32599 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001: Bootstrap complete.
I20260809 08:54:44.986299 32599 raft_consensus.cc:348] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b23f9ebbe3384808a243440a02c63001" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 37419 } }
I20260809 08:54:44.986541 32599 raft_consensus.cc:729] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: b23f9ebbe3384808a243440a02c63001, State: Initialized, Role: FOLLOWER
I20260809 08:54:44.986747 32599 consensus_queue.cc:260] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b23f9ebbe3384808a243440a02c63001" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 37419 } }
I20260809 08:54:44.986845 32599 raft_consensus.cc:388] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:54:44.986900 32599 raft_consensus.cc:482] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:54:44.986949 32599 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [term 2 FOLLOWER]: Advancing to term 3
I20260809 08:54:44.987910 32599 raft_consensus.cc:504] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b23f9ebbe3384808a243440a02c63001" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 37419 } }
I20260809 08:54:44.988030 32599 leader_election.cc:302] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b23f9ebbe3384808a243440a02c63001; no voters:
I20260809 08:54:44.988261 32599 leader_election.cc:288] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 08:54:44.988296 32604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 08:54:44.988408 32604 raft_consensus.cc:686] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [term 3 LEADER]: Becoming Leader. State: Replica: b23f9ebbe3384808a243440a02c63001, State: Running, Role: LEADER
I20260809 08:54:44.988510 32604 consensus_queue.cc:237] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b23f9ebbe3384808a243440a02c63001" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 37419 } }
I20260809 08:54:44.988541 32599 sys_catalog.cc:564] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:54:44.988832 32605 sys_catalog.cc:455] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "b23f9ebbe3384808a243440a02c63001" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b23f9ebbe3384808a243440a02c63001" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 37419 } } }
I20260809 08:54:44.988924 32605 sys_catalog.cc:458] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [sys.catalog]: This master's current role is: LEADER
I20260809 08:54:44.989092 32607 sys_catalog.cc:455] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b23f9ebbe3384808a243440a02c63001. Latest consensus state: current_term: 3 leader_uuid: "b23f9ebbe3384808a243440a02c63001" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b23f9ebbe3384808a243440a02c63001" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 37419 } } }
I20260809 08:54:44.989148 32607 sys_catalog.cc:458] T 00000000000000000000000000000000 P b23f9ebbe3384808a243440a02c63001 [sys.catalog]: This master's current role is: LEADER
I20260809 08:54:44.989609 32610 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:54:44.990379 32610 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:54:44.990481 32610 catalog_manager.cc:1259] Loaded cluster ID: 1e6951e211704a35ade02a0bb7a08b54
I20260809 08:54:44.990514 32610 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 08:54:44.991003 32610 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f7675b439ad google::LogMessage::Fail() at ??:0
@ 0x7f7675b4688a google::LogMessage::SendToLog() at ??:0
@ 0x7f7675b434de google::LogMessage::Flush() at ??:0
@ 0x7f7675b44a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f767da640f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f767da6d8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f7677a110d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f7677a06954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f767886a6db start_thread at ??:0
@ 0x7f767464161f clone at ??:0
W20260809 08:54:45.071985 32601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:54:45.072269 32601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:54:45.072306 32601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:54:45.073464 32601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:54:45.073534 32601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:54:45.073581 32601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.1
I20260809 08:54:45.075054 32601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:36087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.1:43327
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.154.1
--webserver_port=34047
--tserver_master_addrs=127.8.154.62:37419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.1
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:54:45.075320 32601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:54:45.075558 32601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 08:54:45.080710 32601 server_base.cc:1034] running on GCE node
W20260809 08:54:45.080822 32652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:45.080760 32651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:45.080845 32656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:54:45.081300 32601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:54:45.081566 32601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:54:45.082727 32601 hybrid_clock.cc:648] HybridClock initialized: now 1786265685082710 us; error 32 us; skew 500 ppm
Aug 09 08:54:45 dist-test-slave-t3q3 krb5kdc[30877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265685, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:54:45.085534 32601 init.cc:377] Logged in from keytab as kudu/127.8.154.1@KRBTEST.COM (short username kudu)
I20260809 08:54:45.086038 32601 webserver.cc:460] Webserver started at http://127.8.154.1:34047/ using document root <none> and password file <none>
I20260809 08:54:45.086275 32601 fs_manager.cc:359] Metadata directory not provided
I20260809 08:54:45.086345 32601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:54:45.087702 32601 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:54:45.088478 32670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:45.088636 32601 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 08:54:45.088742 32601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/wal
uuid: "4715c949a24b43a3a679777781f0c09b"
format_stamp: "Formatted at 2026-08-09 08:54:43 on dist-test-slave-t3q3"
server_key: "d76fc1bc985111a550198fab0c5f36a8"
server_key_iv: "7f3993954bbffad082306b0d14120418"
server_key_version: "encryptionkey@0"
I20260809 08:54:45.089074 32601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:54:45.104931 32601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:54:45.107245 32601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:54:45.107360 32601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:54:45.107614 32601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:54:45.108119 32679 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 08:54:45.109776 32601 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 08:54:45.109830 32601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 08:54:45.109856 32601 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 08:54:45.110538 32601 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 08:54:45.110585 32601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 08:54:45.110663 32679 tablet_bootstrap.cc:492] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b: Bootstrap starting.
I20260809 08:54:45.118602 32601 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.1:43327
I20260809 08:54:45.118906 374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.1:43327 every 8 connection(s)
I20260809 08:54:45.119181 32601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
I20260809 08:54:45.122314 32679 log.cc:824] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b: Log is configured to *not* fsync() on all Append() calls
I20260809 08:54:45.122409 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 32601
I20260809 08:54:45.122766 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.2:43885
--local_ip_for_outbound_sockets=127.8.154.2
--tserver_master_addrs=127.8.154.62:37419
--webserver_port=38757
--webserver_interface=127.8.154.2
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:36087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:54:45.124707 32679 tablet_bootstrap.cc:492] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 08:54:45.125031 32679 tablet_bootstrap.cc:492] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b: Bootstrap complete.
I20260809 08:54:45.126261 32679 ts_tablet_manager.cc:1397] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.008s
I20260809 08:54:45.128216 32679 raft_consensus.cc:348] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8561daa539d243618585ad07d97c21cb" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 35211 } } peers { permanent_uuid: "0201a5d9a2104dc397da46d707c50b73" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43885 } } peers { permanent_uuid: "4715c949a24b43a3a679777781f0c09b" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43327 } }
I20260809 08:54:45.128636 32679 raft_consensus.cc:729] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4715c949a24b43a3a679777781f0c09b, State: Initialized, Role: FOLLOWER
I20260809 08:54:45.128798 32679 consensus_queue.cc:260] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8561daa539d243618585ad07d97c21cb" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 35211 } } peers { permanent_uuid: "0201a5d9a2104dc397da46d707c50b73" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43885 } } peers { permanent_uuid: "4715c949a24b43a3a679777781f0c09b" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43327 } }
I20260809 08:54:45.129132 32679 ts_tablet_manager.cc:1428] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b: Time spent starting tablet: real 0.003s user 0.005s sys 0.000s
W20260809 08:54:45.207700 384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:54:45.207933 384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:54:45.207958 384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:54:45.209510 384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:54:45.209568 384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:54:45.209599 384 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.2
I20260809 08:54:45.211786 384 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:36087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.2:43885
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.154.2
--webserver_port=38757
--tserver_master_addrs=127.8.154.62:37419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.2
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:54:45.213709 384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:54:45.214044 384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:54:45.220135 391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:45.220142 397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:45.220342 392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:54:45.220363 384 server_base.cc:1034] running on GCE node
I20260809 08:54:45.220588 384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:54:45.220860 384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
W20260809 08:54:45.221448 379 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:54:45.120027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.1:39221)
0809 08:54:45.120273 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:45.120326 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 08:54:45.121142 (+ 816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:54:45.221294 (+100152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":142,"threads_started":1}
W20260809 08:54:45.221674 375 heartbeater.cc:644] Failed to heartbeat to 127.8.154.62:37419 (0 consecutive failures): Network error: Failed to ping master at 127.8.154.62:37419: Client connection negotiation failed: client connection to 127.8.154.62:37419: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 08:54:45.222020 379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:45.221884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.1:51025)
0809 08:54:45.221934 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:45.221971 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 08:54:45.222025 384 hybrid_clock.cc:648] HybridClock initialized: now 1786265685222003 us; error 38 us; skew 500 ppm
I20260809 08:54:45.222424 379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:45.222318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.1:40619)
0809 08:54:45.222363 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:45.222394 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 08:54:45.222543 375 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 08:54:45 dist-test-slave-t3q3 krb5kdc[30877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265685, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:54:45.224664 384 init.cc:377] Logged in from keytab as kudu/127.8.154.2@KRBTEST.COM (short username kudu)
I20260809 08:54:45.225109 384 webserver.cc:460] Webserver started at http://127.8.154.2:38757/ using document root <none> and password file <none>
I20260809 08:54:45.225337 384 fs_manager.cc:359] Metadata directory not provided
I20260809 08:54:45.225383 384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:54:45.226713 384 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:54:45.227361 414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:45.227562 384 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 08:54:45.227655 384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/wal
uuid: "0201a5d9a2104dc397da46d707c50b73"
format_stamp: "Formatted at 2026-08-09 08:54:43 on dist-test-slave-t3q3"
server_key: "1ffd114d8b75f47462163f7358c26adf"
server_key_iv: "11db7ede9c150dc23c774fa4925a19d0"
server_key_version: "encryptionkey@0"
I20260809 08:54:45.227993 384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:54:45.242995 384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:54:45.245011 384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:54:45.245177 384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:54:45.245395 384 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:54:45.245711 384 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:54:45.245772 384 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:45.245812 384 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:54:45.245839 384 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:45.251758 384 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.2:43885
I20260809 08:54:45.251833 576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.2:43885 every 8 connection(s)
I20260809 08:54:45.252228 384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
I20260809 08:54:45.253319 579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:45.252884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.2:57189)
0809 08:54:45.253054 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:45.253163 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":73,"threads_started":1}
W20260809 08:54:45.253865 577 heartbeater.cc:644] Failed to heartbeat to 127.8.154.62:37419 (0 consecutive failures): Network error: Failed to ping master at 127.8.154.62:37419: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
I20260809 08:54:45.254143 579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:45.254053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.2:41145)
0809 08:54:45.254099 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:45.254127 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 08:54:45.254393 579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:45.254343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.2:51843)
0809 08:54:45.254363 (+ 20us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:45.254381 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260809 08:54:45.254495 577 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 08:54:45.260171 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 384
I20260809 08:54:45.260426 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.3:35211
--local_ip_for_outbound_sockets=127.8.154.3
--tserver_master_addrs=127.8.154.62:37419
--webserver_port=46567
--webserver_interface=127.8.154.3
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:36087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:54:45.345149 580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:54:45.345359 580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:54:45.345381 580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:54:45.346544 580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:54:45.346586 580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:54:45.346612 580 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.3
I20260809 08:54:45.348117 580 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:36087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.3:35211
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.154.3
--webserver_port=46567
--tserver_master_addrs=127.8.154.62:37419
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.3
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:54:45.348366 580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:54:45.348605 580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:54:45.353760 590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:45.353760 588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:54:45.353771 586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:54:45.354017 580 server_base.cc:1034] running on GCE node
I20260809 08:54:45.354221 580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:54:45.354415 580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:54:45.355564 580 hybrid_clock.cc:648] HybridClock initialized: now 1786265685355552 us; error 27 us; skew 500 ppm
Aug 09 08:54:45 dist-test-slave-t3q3 krb5kdc[30877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265685, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:54:45.358227 580 init.cc:377] Logged in from keytab as kudu/127.8.154.3@KRBTEST.COM (short username kudu)
I20260809 08:54:45.358703 580 webserver.cc:460] Webserver started at http://127.8.154.3:46567/ using document root <none> and password file <none>
I20260809 08:54:45.358932 580 fs_manager.cc:359] Metadata directory not provided
I20260809 08:54:45.359000 580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:54:45.360354 580 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:54:45.360944 602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:45.361119 580 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:54:45.361227 580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/wal
uuid: "8561daa539d243618585ad07d97c21cb"
format_stamp: "Formatted at 2026-08-09 08:54:43 on dist-test-slave-t3q3"
server_key: "0d8a25b2dda597ccbd3c810d13cce2d2"
server_key_iv: "23abf76242ce79a756dde566f53d4e9b"
server_key_version: "encryptionkey@0"
I20260809 08:54:45.361567 580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:54:45.368764 580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:54:45.370571 580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:54:45.370743 580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:54:45.370963 580 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:54:45.371277 580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:54:45.371342 580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:45.371384 580 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:54:45.371425 580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:54:45.377619 580 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.3:35211
I20260809 08:54:45.377668 777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.3:35211 every 8 connection(s)
I20260809 08:54:45.378029 580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
I20260809 08:54:45.379077 783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:45.378690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.3:52793)
0809 08:54:45.378862 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:45.378934 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":86,"threads_started":1}
W20260809 08:54:45.379487 779 heartbeater.cc:644] Failed to heartbeat to 127.8.154.62:37419 (0 consecutive failures): Network error: Failed to ping master at 127.8.154.62:37419: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
I20260809 08:54:45.379782 783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:45.379687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.3:50649)
0809 08:54:45.379736 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:45.379765 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 08:54:45.380024 783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:45.379974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.3:52775)
0809 08:54:45.379996 (+ 22us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:45.380013 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260809 08:54:45.380143 779 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 08:54:45.386348 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 580
I20260809 08:54:46.223191 786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:46.222930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.1:41631)
0809 08:54:46.223074 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:46.223123 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":66,"threads_started":1}
I20260809 08:54:46.255039 787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:46.254827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.2:33089)
0809 08:54:46.254963 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:46.254992 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":86,"threads_started":1}
I20260809 08:54:46.380945 788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:46.380565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.3:57335)
0809 08:54:46.380791 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:46.380850 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":116,"threads_started":1}
I20260809 08:54:46.988464 789 raft_consensus.cc:482] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:54:46.988626 789 raft_consensus.cc:504] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8561daa539d243618585ad07d97c21cb" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 35211 } } peers { permanent_uuid: "0201a5d9a2104dc397da46d707c50b73" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43885 } } peers { permanent_uuid: "4715c949a24b43a3a679777781f0c09b" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43327 } }
I20260809 08:54:46.988986 789 leader_election.cc:288] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8561daa539d243618585ad07d97c21cb (127.8.154.3:35211), 0201a5d9a2104dc397da46d707c50b73 (127.8.154.2:43885)
Aug 09 08:54:46 dist-test-slave-t3q3 krb5kdc[30877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265685, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for kudu/127.8.154.2@KRBTEST.COM
Aug 09 08:54:46 dist-test-slave-t3q3 krb5kdc[30877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265685, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for kudu/127.8.154.3@KRBTEST.COM
I20260809 08:54:47.003027 792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:46.989155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:43885 (local address 127.8.154.1:37117)
0809 08:54:46.989322 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:46.989333 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:54:46.989393 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:54:46.990409 (+ 1016us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:54:46.990410 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:54:46.990640 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:54:46.990807 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:46.990812 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:46.992143 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:46.992144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:46.992960 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:46.992963 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:46.993619 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:46.993621 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:46.993733 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:46.997570 (+ 3837us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:54:46.997574 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:54:46.999143 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:54:47.001240 (+ 2097us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:47.001247 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:47.001249 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:47.001544 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:47.001805 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:47.001806 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:47.001806 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:47.001937 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:47.002505 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:54:47.002512 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:54:47.002652 (+ 140us) client_negotiation.cc:769] Sending connection context
0809 08:54:47.002742 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 08:54:47.002902 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":53,"thread_start_us":86,"threads_started":1}
I20260809 08:54:47.003103 795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:46.989316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:37117 (local address 127.8.154.2:43885)
0809 08:54:46.989492 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:46.989500 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:46.989519 (+ 19us) server_negotiation.cc:407] Connection header received
0809 08:54:46.990055 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:46.990058 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:46.990117 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:46.990272 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:54:46.990953 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:46.992017 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:46.993111 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:46.993502 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:46.993628 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:46.999264 (+ 5636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:54:46.999268 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:54:46.999268 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:54:46.999278 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:54:47.001154 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:47.001653 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:47.001656 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:47.001656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:47.001736 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:47.002007 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:47.002008 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:47.002009 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:47.002386 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:54:47.002475 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:47.002830 (+ 355us) server_negotiation.cc:299] Negotiation successful
0809 08:54:47.002991 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":81,"threads_started":1}
I20260809 08:54:47.003334 794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:46.989255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:51687 (local address 127.8.154.3:35211)
0809 08:54:46.989440 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 08:54:46.989453 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 08:54:46.989480 (+ 27us) server_negotiation.cc:407] Connection header received
0809 08:54:46.990055 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:54:46.990058 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:54:46.990119 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:54:46.990272 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:54:46.990934 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:46.991920 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:46.993073 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:54:46.993493 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:54:46.993603 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:46.999747 (+ 6144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:54:46.999751 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:54:46.999752 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:54:46.999765 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:54:47.001750 (+ 1985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:47.002164 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:47.002167 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:47.002167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:47.002272 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:54:47.002504 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:54:47.002506 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:54:47.002506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:54:47.002767 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:54:47.002853 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 08:54:47.003049 (+ 196us) server_negotiation.cc:299] Negotiation successful
0809 08:54:47.003264 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":57,"threads_started":1}
I20260809 08:54:47.003177 791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:46.989130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:35211 (local address 127.8.154.1:51687)
0809 08:54:46.989243 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:46.989254 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:54:46.989320 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:54:46.990308 (+ 988us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:54:46.990310 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:54:46.990611 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:54:46.990806 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:46.990812 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:46.992045 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:46.992046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:46.992963 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:54:46.992966 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:54:46.993585 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:54:46.993587 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:54:46.993744 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:54:46.997543 (+ 3799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:54:46.997555 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:54:46.999637 (+ 2082us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:54:47.001843 (+ 2206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:47.001844 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:47.001845 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:47.002067 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:47.002347 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:54:47.002348 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:54:47.002348 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:54:47.002421 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:54:47.002852 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:54:47.002853 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:54:47.002898 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 08:54:47.002971 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 08:54:47.003055 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":50,"threads_started":1}
W20260809 08:54:47.004531 32671 leader_election.cc:341] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0201a5d9a2104dc397da46d707c50b73 (127.8.154.2:43885): Not found: Tablet not found: 2dbe195ea64c479fb1978c24bcf867f3
W20260809 08:54:47.004741 32674 leader_election.cc:341] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8561daa539d243618585ad07d97c21cb (127.8.154.3:35211): Not found: Tablet not found: 2dbe195ea64c479fb1978c24bcf867f3
I20260809 08:54:47.004792 32674 leader_election.cc:302] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4715c949a24b43a3a679777781f0c09b; no voters: 0201a5d9a2104dc397da46d707c50b73, 8561daa539d243618585ad07d97c21cb
I20260809 08:54:47.004904 789 raft_consensus.cc:2726] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 08:54:47.223804 791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:47.223642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.1:52181)
0809 08:54:47.223715 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:47.223756 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 08:54:47.255722 797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:47.255476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.2:42815)
0809 08:54:47.255622 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:47.255674 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":101,"threads_started":1}
I20260809 08:54:47.381680 798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:47.381452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.3:34413)
0809 08:54:47.381593 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:47.381639 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":77,"threads_started":1}
I20260809 08:54:48.224550 799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:48.224324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.1:38981)
0809 08:54:48.224467 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:48.224500 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":79,"threads_started":1}
I20260809 08:54:48.256387 800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:48.256184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.2:52599)
0809 08:54:48.256314 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:48.256348 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":74,"threads_started":1}
I20260809 08:54:48.382467 801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:48.382239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.3:50073)
0809 08:54:48.382387 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:48.382424 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":78,"threads_started":1}
I20260809 08:54:49.115185 802 raft_consensus.cc:482] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:54:49.115283 802 raft_consensus.cc:504] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8561daa539d243618585ad07d97c21cb" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 35211 } } peers { permanent_uuid: "0201a5d9a2104dc397da46d707c50b73" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43885 } } peers { permanent_uuid: "4715c949a24b43a3a679777781f0c09b" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43327 } }
I20260809 08:54:49.115510 802 leader_election.cc:288] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8561daa539d243618585ad07d97c21cb (127.8.154.3:35211), 0201a5d9a2104dc397da46d707c50b73 (127.8.154.2:43885)
W20260809 08:54:49.115931 32674 leader_election.cc:341] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8561daa539d243618585ad07d97c21cb (127.8.154.3:35211): Not found: Tablet not found: 2dbe195ea64c479fb1978c24bcf867f3
W20260809 08:54:49.116006 32671 leader_election.cc:341] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0201a5d9a2104dc397da46d707c50b73 (127.8.154.2:43885): Not found: Tablet not found: 2dbe195ea64c479fb1978c24bcf867f3
I20260809 08:54:49.116142 32671 leader_election.cc:302] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4715c949a24b43a3a679777781f0c09b; no voters: 0201a5d9a2104dc397da46d707c50b73, 8561daa539d243618585ad07d97c21cb
I20260809 08:54:49.116235 802 raft_consensus.cc:2726] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 08:54:49.225303 803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:49.225064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.1:37159)
0809 08:54:49.225213 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:49.225255 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":71,"threads_started":1}
I20260809 08:54:49.257077 804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:49.256853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.2:37041)
0809 08:54:49.256981 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:49.257021 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":63,"threads_started":1}
I20260809 08:54:49.383244 808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:49.382981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.3:47977)
0809 08:54:49.383132 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:49.383175 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":82,"threads_started":1}
I20260809 08:54:50.226081 809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:50.225841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.1:36119)
0809 08:54:50.226001 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:50.226035 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":101,"threads_started":1}
I20260809 08:54:50.257752 810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:50.257528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.2:54335)
0809 08:54:50.257674 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:50.257706 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":56,"threads_started":1}
I20260809 08:54:50.383921 811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:50.383705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.3:55223)
0809 08:54:50.383844 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:50.383877 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":79,"threads_started":1}
I20260809 08:54:51.226789 812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:51.226562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.1:36197)
0809 08:54:51.226706 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:51.226745 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":64,"threads_started":1}
I20260809 08:54:51.258373 814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:51.258162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.2:57531)
0809 08:54:51.258301 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:51.258330 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":84,"threads_started":1}
I20260809 08:54:51.384620 816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:51.384389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.3:43821)
0809 08:54:51.384530 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:51.384569 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":62,"threads_started":1}
I20260809 08:54:52.227540 817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:52.227291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.1:56073)
0809 08:54:52.227421 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:52.227477 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":65,"threads_started":1}
I20260809 08:54:52.259068 818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:52.258846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.2:54287)
0809 08:54:52.258985 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:52.259023 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":68,"threads_started":1}
I20260809 08:54:52.283123 819 raft_consensus.cc:482] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:54:52.283236 819 raft_consensus.cc:504] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8561daa539d243618585ad07d97c21cb" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 35211 } } peers { permanent_uuid: "0201a5d9a2104dc397da46d707c50b73" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43885 } } peers { permanent_uuid: "4715c949a24b43a3a679777781f0c09b" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43327 } }
I20260809 08:54:52.283463 819 leader_election.cc:288] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8561daa539d243618585ad07d97c21cb (127.8.154.3:35211), 0201a5d9a2104dc397da46d707c50b73 (127.8.154.2:43885)
W20260809 08:54:52.283939 32674 leader_election.cc:341] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8561daa539d243618585ad07d97c21cb (127.8.154.3:35211): Not found: Tablet not found: 2dbe195ea64c479fb1978c24bcf867f3
W20260809 08:54:52.284022 32671 leader_election.cc:341] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0201a5d9a2104dc397da46d707c50b73 (127.8.154.2:43885): Not found: Tablet not found: 2dbe195ea64c479fb1978c24bcf867f3
I20260809 08:54:52.284053 32674 leader_election.cc:302] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4715c949a24b43a3a679777781f0c09b; no voters: 0201a5d9a2104dc397da46d707c50b73, 8561daa539d243618585ad07d97c21cb
I20260809 08:54:52.284165 819 raft_consensus.cc:2726] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 08:54:52.385422 820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:52.385156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.3:34793)
0809 08:54:52.385303 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:52.385366 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":68,"threads_started":1}
I20260809 08:54:53.228359 821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:53.228121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.1:52253)
0809 08:54:53.228270 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:53.228312 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":69,"threads_started":1}
I20260809 08:54:53.259812 822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:53.259578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.2:37177)
0809 08:54:53.259722 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:53.259765 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":68,"threads_started":1}
I20260809 08:54:53.386226 823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:53.385958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.3:39619)
0809 08:54:53.386134 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:53.386176 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":96,"threads_started":1}
I20260809 08:54:54.229141 824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:54.228933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.1:38191)
0809 08:54:54.229059 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:54.229096 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":65,"threads_started":1}
I20260809 08:54:54.260566 826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:54.260352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.2:45667)
0809 08:54:54.260486 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:54.260522 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":82,"threads_started":1}
I20260809 08:54:54.386909 828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:54.386680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.3:34807)
0809 08:54:54.386825 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:54.386866 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":81,"threads_started":1}
I20260809 08:54:54.732824 829 raft_consensus.cc:482] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:54:54.732939 829 raft_consensus.cc:504] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8561daa539d243618585ad07d97c21cb" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 35211 } } peers { permanent_uuid: "0201a5d9a2104dc397da46d707c50b73" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43885 } } peers { permanent_uuid: "4715c949a24b43a3a679777781f0c09b" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43327 } }
I20260809 08:54:54.733188 829 leader_election.cc:288] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8561daa539d243618585ad07d97c21cb (127.8.154.3:35211), 0201a5d9a2104dc397da46d707c50b73 (127.8.154.2:43885)
W20260809 08:54:54.733537 32674 leader_election.cc:341] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8561daa539d243618585ad07d97c21cb (127.8.154.3:35211): Not found: Tablet not found: 2dbe195ea64c479fb1978c24bcf867f3
W20260809 08:54:54.733604 32671 leader_election.cc:341] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0201a5d9a2104dc397da46d707c50b73 (127.8.154.2:43885): Not found: Tablet not found: 2dbe195ea64c479fb1978c24bcf867f3
I20260809 08:54:54.733641 32671 leader_election.cc:302] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4715c949a24b43a3a679777781f0c09b; no voters: 0201a5d9a2104dc397da46d707c50b73, 8561daa539d243618585ad07d97c21cb
I20260809 08:54:54.733709 829 raft_consensus.cc:2726] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 08:54:55.229902 830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:55.229670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.1:60463)
0809 08:54:55.229810 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:55.229854 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":63,"threads_started":1}
I20260809 08:54:55.261266 831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:55.261071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.2:59865)
0809 08:54:55.261188 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:55.261224 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":59,"threads_started":1}
I20260809 08:54:55.387645 832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:55.387433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.3:58575)
0809 08:54:55.387556 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:55.387596 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":63,"threads_started":1}
I20260809 08:54:56.230780 833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:56.230539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.1:49425)
0809 08:54:56.230666 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:56.230725 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":67,"threads_started":1}
I20260809 08:54:56.262094 835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:56.261867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.2:37961)
0809 08:54:56.262012 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:56.262042 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
I20260809 08:54:56.388451 836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:56.388215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.3:51391)
0809 08:54:56.388360 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:56.388401 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":68,"threads_started":1}
I20260809 08:54:57.233060 837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:57.231493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.1:50457)
0809 08:54:57.232942 (+ 1449us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:57.232999 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1426,"thread_start_us":1395,"threads_started":1}
I20260809 08:54:57.262977 839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:57.262743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.2:45201)
0809 08:54:57.262885 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:57.262929 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":62,"threads_started":1}
I20260809 08:54:57.389192 841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:57.388952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.3:58015)
0809 08:54:57.389090 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:57.389143 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":66,"threads_started":1}
I20260809 08:54:58.233768 842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:58.233534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.1:47639)
0809 08:54:58.233683 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:58.233722 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":67,"threads_started":1}
I20260809 08:54:58.263690 843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:58.263485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.2:44049)
0809 08:54:58.263612 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:58.263648 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":66,"threads_started":1}
I20260809 08:54:58.390009 844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:58.389776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.3:34699)
0809 08:54:58.389924 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:58.389962 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":73,"threads_started":1}
I20260809 08:54:59.234635 846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:59.234379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.1:44289)
0809 08:54:59.234535 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:59.234585 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":65,"threads_started":1}
I20260809 08:54:59.264650 848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:59.264367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.2:34235)
0809 08:54:59.264550 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:59.264601 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":99,"threads_started":1}
I20260809 08:54:59.390704 852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:54:59.390474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.3:34643)
0809 08:54:59.390623 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 08:54:59.390658 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":72,"threads_started":1}
I20260809 08:55:00.098222 853 raft_consensus.cc:482] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:55:00.098366 853 raft_consensus.cc:504] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8561daa539d243618585ad07d97c21cb" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 35211 } } peers { permanent_uuid: "0201a5d9a2104dc397da46d707c50b73" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43885 } } peers { permanent_uuid: "4715c949a24b43a3a679777781f0c09b" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 43327 } }
I20260809 08:55:00.098614 853 leader_election.cc:288] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8561daa539d243618585ad07d97c21cb (127.8.154.3:35211), 0201a5d9a2104dc397da46d707c50b73 (127.8.154.2:43885)
W20260809 08:55:00.099026 32674 leader_election.cc:341] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8561daa539d243618585ad07d97c21cb (127.8.154.3:35211): Not found: Tablet not found: 2dbe195ea64c479fb1978c24bcf867f3
W20260809 08:55:00.099152 32671 leader_election.cc:341] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0201a5d9a2104dc397da46d707c50b73 (127.8.154.2:43885): Not found: Tablet not found: 2dbe195ea64c479fb1978c24bcf867f3
I20260809 08:55:00.099227 32671 leader_election.cc:302] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4715c949a24b43a3a679777781f0c09b; no voters: 0201a5d9a2104dc397da46d707c50b73, 8561daa539d243618585ad07d97c21cb
I20260809 08:55:00.099321 853 raft_consensus.cc:2726] T 2dbe195ea64c479fb1978c24bcf867f3 P 4715c949a24b43a3a679777781f0c09b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 08:55:00.235580 854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:00.235330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.1:33343)
0809 08:55:00.235489 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:00.235525 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":93,"threads_started":1}
I20260809 08:55:00.265363 855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:00.265161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.2:35835)
0809 08:55:00.265289 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:00.265323 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":62,"threads_started":1}
I20260809 08:55:00.391615 859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:00.391345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:37419 (local address 127.8.154.3:43167)
0809 08:55:00.391515 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:00.391565 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:37419: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":81,"threads_started":1}
I20260809 08:55:00.392096 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 32601
I20260809 08:55:00.402549 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 384
I20260809 08:55:00.416512 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 580
2026-08-09T08:55:00Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18557 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:55:00 dist-test-slave-t3q3 krb5kdc[862](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:55:00 dist-test-slave-t3q3 krb5kdc[862](info): set up 2 sockets
Aug 09 08:55:00 dist-test-slave-t3q3 krb5kdc[862](info): commencing operation
krb5kdc: starting...
W20260809 08:55:01.686725 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:55:01 dist-test-slave-t3q3 krb5kdc[862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:55:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:55:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
I20260809 08:55:01.812379 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:46845
--webserver_interface=127.8.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:39543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:46845
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:55:01.896303 879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:55:01.896524 879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:55:01.896548 879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:55:01.897861 879 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 08:55:01.897904 879 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 08:55:01.897917 879 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 08:55:01.897929 879 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:55:01.897943 879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:55:01.897990 879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:55:01.898015 879 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:55:01.898038 879 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:55:01.899597 879 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:39543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:46845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:46845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:55:01.899892 879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:55:01.900169 879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:55:01.905833 887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:55:01.905839 889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:55:01.905793 879 server_base.cc:1034] running on GCE node
W20260809 08:55:01.905786 886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:55:01.906368 879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:55:01.906678 879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:55:01.907847 879 hybrid_clock.cc:648] HybridClock initialized: now 1786265701907832 us; error 29 us; skew 500 ppm
Aug 09 08:55:01 dist-test-slave-t3q3 krb5kdc[862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265701, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:55:01.913884 879 init.cc:377] Logged in from keytab as kudu/127.8.154.62@KRBTEST.COM (short username kudu)
I20260809 08:55:01.914341 879 webserver.cc:460] Webserver started at http://127.8.154.62:43855/ using document root <none> and password file <none>
I20260809 08:55:01.914592 879 fs_manager.cc:359] Metadata directory not provided
I20260809 08:55:01.914664 879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:55:01.914880 879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:55:01.915984 879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "75ab210e8bd44fafb02a3561a90adb10"
format_stamp: "Formatted at 2026-08-09 08:55:01 on dist-test-slave-t3q3"
server_key: "8b389bee1bba5c18ca13141e7ece957f"
server_key_iv: "3b3f285f6511c6ee661d89f023f36d80"
server_key_version: "encryptionkey@0"
I20260809 08:55:01.916424 879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "75ab210e8bd44fafb02a3561a90adb10"
format_stamp: "Formatted at 2026-08-09 08:55:01 on dist-test-slave-t3q3"
server_key: "8b389bee1bba5c18ca13141e7ece957f"
server_key_iv: "3b3f285f6511c6ee661d89f023f36d80"
server_key_version: "encryptionkey@0"
I20260809 08:55:01.917794 879 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:55:01.918519 901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:55:01.918707 879 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:55:01.918826 879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "75ab210e8bd44fafb02a3561a90adb10"
format_stamp: "Formatted at 2026-08-09 08:55:01 on dist-test-slave-t3q3"
server_key: "8b389bee1bba5c18ca13141e7ece957f"
server_key_iv: "3b3f285f6511c6ee661d89f023f36d80"
server_key_version: "encryptionkey@0"
I20260809 08:55:01.918900 879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:55:01.932893 879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:55:01.934690 879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:55:01.934856 879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:55:01.938863 879 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:46845
I20260809 08:55:01.938901 986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:46845 every 8 connection(s)
I20260809 08:55:01.939291 879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:55:01.940271 987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:55:01.942567 987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10: Bootstrap starting.
I20260809 08:55:01.943200 987 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10: Neither blocks nor log segments found. Creating new log.
I20260809 08:55:01.943506 987 log.cc:824] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10: Log is configured to *not* fsync() on all Append() calls
I20260809 08:55:01.944288 987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10: No bootstrap required, opened a new log
I20260809 08:55:01.945871 987 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75ab210e8bd44fafb02a3561a90adb10" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46845 } }
I20260809 08:55:01.946012 987 raft_consensus.cc:374] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:55:01.946035 987 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75ab210e8bd44fafb02a3561a90adb10, State: Initialized, Role: FOLLOWER
I20260809 08:55:01.946244 987 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75ab210e8bd44fafb02a3561a90adb10" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46845 } }
I20260809 08:55:01.946334 987 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:55:01.946389 987 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:55:01.946441 987 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:55:01.947016 987 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75ab210e8bd44fafb02a3561a90adb10" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46845 } }
I20260809 08:55:01.947108 987 leader_election.cc:302] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75ab210e8bd44fafb02a3561a90adb10; no voters:
I20260809 08:55:01.947314 987 leader_election.cc:288] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:55:01.947358 990 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:55:01.947544 990 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [term 1 LEADER]: Becoming Leader. State: Replica: 75ab210e8bd44fafb02a3561a90adb10, State: Running, Role: LEADER
I20260809 08:55:01.947594 987 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:55:01.947640 990 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75ab210e8bd44fafb02a3561a90adb10" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46845 } }
I20260809 08:55:01.948252 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 879
I20260809 08:55:01.948385 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:55:01.948606 8808 external_mini_cluster.cc:1428] Setting key a112b1c431907632e0393e3454e4bf55
I20260809 08:55:01.949250 994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75ab210e8bd44fafb02a3561a90adb10. Latest consensus state: current_term: 1 leader_uuid: "75ab210e8bd44fafb02a3561a90adb10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75ab210e8bd44fafb02a3561a90adb10" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46845 } } }
I20260809 08:55:01.949283 993 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "75ab210e8bd44fafb02a3561a90adb10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75ab210e8bd44fafb02a3561a90adb10" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46845 } } }
I20260809 08:55:01.949448 993 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [sys.catalog]: This master's current role is: LEADER
I20260809 08:55:01.949430 994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [sys.catalog]: This master's current role is: LEADER
W20260809 08:55:01.950206 1015 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:55:01.950263 1015 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:55:01.950317 1018 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:55:01.950452 1018 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:55:01.951870 1018 catalog_manager.cc:1347] Generated new cluster ID: 500c1f97dbb24587b8b25ae17323d610
I20260809 08:55:01.951926 1018 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 08:55:01 dist-test-slave-t3q3 krb5kdc[862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265701, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:55:01.959065 1018 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:55:01.959714 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:01.949098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37646 (local address 127.8.154.62:46845)
0809 08:55:01.949545 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0809 08:55:01.949555 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:55:01.949592 (+ 37us) server_negotiation.cc:407] Connection header received
0809 08:55:01.949729 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:55:01.949736 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:55:01.949797 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:55:01.949984 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:55:01.950775 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:01.952156 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:01.952939 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:01.953374 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:01.953498 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:01.956134 (+ 2636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:55:01.956138 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:55:01.956139 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:55:01.956150 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:55:01.957848 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:55:01.958240 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:55:01.958244 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:55:01.958244 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:55:01.958352 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:55:01.958594 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:55:01.958595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:55:01.958596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:55:01.958887 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:55:01.959036 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 08:55:01.959248 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 08:55:01.959394 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":63,"threads_started":1}
I20260809 08:55:01.959722 1018 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:55:01.963239 1018 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10: Generated new TSK 0
I20260809 08:55:01.963407 1018 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab.
Entry for principal kudu/127.8.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab.
I20260809 08:55:01.995782 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.1:0
--local_ip_for_outbound_sockets=127.8.154.1
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:46845
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:39543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:55:02.083622 1026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:55:02.083818 1026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:55:02.083842 1026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:55:02.085032 1026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:55:02.085072 1026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:55:02.085098 1026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.1
I20260809 08:55:02.086531 1026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:39543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:46845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.1
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:55:02.086771 1026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:55:02.087009 1026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:55:02.092114 1038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:55:02.092126 1035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:55:02.092149 1036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:55:02.092634 1026 server_base.cc:1034] running on GCE node
I20260809 08:55:02.092867 1026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:55:02.093156 1026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:55:02.094329 1026 hybrid_clock.cc:648] HybridClock initialized: now 1786265702094310 us; error 35 us; skew 500 ppm
Aug 09 08:55:02 dist-test-slave-t3q3 krb5kdc[862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265702, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:55:02.097059 1026 init.cc:377] Logged in from keytab as kudu/127.8.154.1@KRBTEST.COM (short username kudu)
I20260809 08:55:02.097594 1026 webserver.cc:460] Webserver started at http://127.8.154.1:43363/ using document root <none> and password file <none>
I20260809 08:55:02.097834 1026 fs_manager.cc:359] Metadata directory not provided
I20260809 08:55:02.097903 1026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:55:02.098085 1026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:55:02.099146 1026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/data/instance:
uuid: "c17f5ac68cd74824828707b8ac593ffc"
format_stamp: "Formatted at 2026-08-09 08:55:02 on dist-test-slave-t3q3"
server_key: "8a084a3118dae1161263b0b7dfb9ff6a"
server_key_iv: "fe473b1bc4075b391181955ffcea863b"
server_key_version: "encryptionkey@0"
I20260809 08:55:02.099534 1026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance:
uuid: "c17f5ac68cd74824828707b8ac593ffc"
format_stamp: "Formatted at 2026-08-09 08:55:02 on dist-test-slave-t3q3"
server_key: "8a084a3118dae1161263b0b7dfb9ff6a"
server_key_iv: "fe473b1bc4075b391181955ffcea863b"
server_key_version: "encryptionkey@0"
I20260809 08:55:02.101121 1026 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260809 08:55:02.101953 1050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:55:02.102201 1026 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:55:02.102304 1026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/wal
uuid: "c17f5ac68cd74824828707b8ac593ffc"
format_stamp: "Formatted at 2026-08-09 08:55:02 on dist-test-slave-t3q3"
server_key: "8a084a3118dae1161263b0b7dfb9ff6a"
server_key_iv: "fe473b1bc4075b391181955ffcea863b"
server_key_version: "encryptionkey@0"
I20260809 08:55:02.102383 1026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:55:02.114085 1026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:55:02.116443 1026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:55:02.116611 1026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:55:02.116861 1026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:55:02.117177 1026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:55:02.117233 1026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:55:02.117276 1026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:55:02.117317 1026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:55:02.123328 1026 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.1:45617
I20260809 08:55:02.123386 1229 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.1:45617 every 8 connection(s)
I20260809 08:55:02.123749 1026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
Aug 09 08:55:02 dist-test-slave-t3q3 krb5kdc[862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265702, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:55:02.132391 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 1026
I20260809 08:55:02.132526 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance
I20260809 08:55:02.132793 8808 external_mini_cluster.cc:1428] Setting key a022601b32f0cb3c38499a9df593d540
I20260809 08:55:02.133719 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:02.124530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:51805 (local address 127.8.154.62:46845)
0809 08:55:02.124601 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 08:55:02.124603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:55:02.124845 (+ 242us) server_negotiation.cc:407] Connection header received
0809 08:55:02.125345 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:55:02.125347 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:55:02.125361 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:55:02.125424 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:55:02.126389 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.126921 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.127943 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.128252 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.128318 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:02.130467 (+ 2149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:55:02.130468 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:55:02.130469 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:55:02.130477 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:55:02.131798 (+ 1321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:55:02.132382 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:55:02.132383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:55:02.132383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:55:02.132444 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:55:02.132830 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:55:02.132830 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:55:02.132831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:55:02.133038 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:55:02.133126 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 08:55:02.133490 (+ 364us) server_negotiation.cc:299] Negotiation successful
0809 08:55:02.133624 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 08:55:02.133850 1234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:02.124522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.1:51805)
0809 08:55:02.124717 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:02.124752 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 08:55:02.125234 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:55:02.125503 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:55:02.125506 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:55:02.126089 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:55:02.126287 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:55:02.126292 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.127011 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.127012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:55:02.127848 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:55:02.127850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.128333 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.128334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:55:02.128438 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:02.128953 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:55:02.128961 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:55:02.130360 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:55:02.131916 (+ 1556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:55:02.131924 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:55:02.131926 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:55:02.132297 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:55:02.132574 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:55:02.132576 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:55:02.132576 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:55:02.132755 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:55:02.133125 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:55:02.133133 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:55:02.133320 (+ 187us) client_negotiation.cc:769] Sending connection context
0809 08:55:02.133442 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 08:55:02.133629 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":112,"threads_started":1}
I20260809 08:55:02.134398 1231 heartbeater.cc:347] Connected to a master server at 127.8.154.62:46845
I20260809 08:55:02.134493 1231 heartbeater.cc:464] Registering TS with master...
I20260809 08:55:02.134701 1231 heartbeater.cc:511] Master 127.8.154.62:46845 requested a full tablet report, sending...
I20260809 08:55:02.135226 935 ts_manager.cc:194] Registered new tserver with Master: c17f5ac68cd74824828707b8ac593ffc (127.8.154.1:45617)
I20260809 08:55:02.135855 935 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.1@KRBTEST.COM'} at 127.8.154.1:51805
WARNING: no policy specified for kudu/127.8.154.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab.
Entry for principal kudu/127.8.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab.
I20260809 08:55:02.158907 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.2:0
--local_ip_for_outbound_sockets=127.8.154.2
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:46845
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:39543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:55:02.242261 1239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:55:02.242509 1239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:55:02.242548 1239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:55:02.243681 1239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:55:02.243752 1239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:55:02.243798 1239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.2
I20260809 08:55:02.245301 1239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:39543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:46845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.2
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:55:02.245581 1239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:55:02.245812 1239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:55:02.251127 1246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:55:02.251139 1247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:55:02.251142 1249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:55:02.251518 1239 server_base.cc:1034] running on GCE node
I20260809 08:55:02.251745 1239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:55:02.251991 1239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:55:02.253135 1239 hybrid_clock.cc:648] HybridClock initialized: now 1786265702253124 us; error 29 us; skew 500 ppm
Aug 09 08:55:02 dist-test-slave-t3q3 krb5kdc[862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265702, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:55:02.255760 1239 init.cc:377] Logged in from keytab as kudu/127.8.154.2@KRBTEST.COM (short username kudu)
I20260809 08:55:02.256285 1239 webserver.cc:460] Webserver started at http://127.8.154.2:40719/ using document root <none> and password file <none>
I20260809 08:55:02.256482 1239 fs_manager.cc:359] Metadata directory not provided
I20260809 08:55:02.256521 1239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:55:02.256610 1239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:55:02.257584 1239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/data/instance:
uuid: "d040e1f52a2743eaaccaa5e30c679477"
format_stamp: "Formatted at 2026-08-09 08:55:02 on dist-test-slave-t3q3"
server_key: "1084e86c51c5bf6ded8911231565fdad"
server_key_iv: "f3b21401e84290c0334f3d68eb6f17ea"
server_key_version: "encryptionkey@0"
I20260809 08:55:02.257885 1239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance:
uuid: "d040e1f52a2743eaaccaa5e30c679477"
format_stamp: "Formatted at 2026-08-09 08:55:02 on dist-test-slave-t3q3"
server_key: "1084e86c51c5bf6ded8911231565fdad"
server_key_iv: "f3b21401e84290c0334f3d68eb6f17ea"
server_key_version: "encryptionkey@0"
I20260809 08:55:02.259120 1239 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:55:02.259821 1259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:55:02.259994 1239 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:55:02.260118 1239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/wal
uuid: "d040e1f52a2743eaaccaa5e30c679477"
format_stamp: "Formatted at 2026-08-09 08:55:02 on dist-test-slave-t3q3"
server_key: "1084e86c51c5bf6ded8911231565fdad"
server_key_iv: "f3b21401e84290c0334f3d68eb6f17ea"
server_key_version: "encryptionkey@0"
I20260809 08:55:02.260211 1239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:55:02.276458 1239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:55:02.278612 1239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:55:02.278792 1239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:55:02.279042 1239 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:55:02.279392 1239 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:55:02.279446 1239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:55:02.279481 1239 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:55:02.279508 1239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:55:02.285550 1239 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.2:43319
I20260809 08:55:02.285653 1428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.2:43319 every 8 connection(s)
I20260809 08:55:02.285967 1239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
Aug 09 08:55:02 dist-test-slave-t3q3 krb5kdc[862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265702, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:55:02.294364 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 1239
I20260809 08:55:02.294492 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance
I20260809 08:55:02.294744 8808 external_mini_cluster.cc:1428] Setting key 3aaec2467bef9547c7a33b093f4fd787
I20260809 08:55:02.295576 1432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:02.286712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.2:47985)
0809 08:55:02.286914 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:02.286951 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 08:55:02.287425 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:55:02.287667 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:55:02.287671 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:55:02.288326 (+ 655us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:55:02.288629 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:55:02.288635 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.289315 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.289315 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:55:02.290125 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:55:02.290127 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.290475 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.290476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:55:02.290597 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:02.291057 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:55:02.291065 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:55:02.292452 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:55:02.293922 (+ 1470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:55:02.293929 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:55:02.293932 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:55:02.294175 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:55:02.294409 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:55:02.294410 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:55:02.294410 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:55:02.294571 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:55:02.294907 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:55:02.294914 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:55:02.295054 (+ 140us) client_negotiation.cc:769] Sending connection context
0809 08:55:02.295177 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 08:55:02.295345 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":113,"threads_started":1}
I20260809 08:55:02.295612 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:02.286745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:47985 (local address 127.8.154.62:46845)
0809 08:55:02.286825 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 08:55:02.286827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:55:02.287046 (+ 219us) server_negotiation.cc:407] Connection header received
0809 08:55:02.287512 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:55:02.287513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:55:02.287524 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:55:02.287579 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:55:02.288726 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.289236 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.290203 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.290397 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.290451 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:02.292563 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:55:02.292564 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:55:02.292564 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:55:02.292572 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:55:02.293832 (+ 1260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:55:02.294270 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:55:02.294271 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:55:02.294271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:55:02.294332 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:55:02.294651 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:55:02.294652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:55:02.294653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:55:02.294767 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:55:02.294874 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 08:55:02.295433 (+ 559us) server_negotiation.cc:299] Negotiation successful
0809 08:55:02.295518 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 08:55:02.296144 1429 heartbeater.cc:347] Connected to a master server at 127.8.154.62:46845
I20260809 08:55:02.296257 1429 heartbeater.cc:464] Registering TS with master...
I20260809 08:55:02.296455 1429 heartbeater.cc:511] Master 127.8.154.62:46845 requested a full tablet report, sending...
I20260809 08:55:02.296854 935 ts_manager.cc:194] Registered new tserver with Master: d040e1f52a2743eaaccaa5e30c679477 (127.8.154.2:43319)
I20260809 08:55:02.297253 935 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.2@KRBTEST.COM'} at 127.8.154.2:47985
WARNING: no policy specified for kudu/127.8.154.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab.
Entry for principal kudu/127.8.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab.
I20260809 08:55:02.322947 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.3:0
--local_ip_for_outbound_sockets=127.8.154.3
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:46845
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:39543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:55:02.406299 1435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:55:02.406500 1435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:55:02.406527 1435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:55:02.407660 1435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:55:02.407702 1435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:55:02.407754 1435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.3
I20260809 08:55:02.409250 1435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:39543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:46845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.3
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:55:02.409499 1435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:55:02.409744 1435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:55:02.414956 1442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:55:02.414974 1447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:55:02.415179 1445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:55:02.415156 1435 server_base.cc:1034] running on GCE node
I20260809 08:55:02.415374 1435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:55:02.415624 1435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:55:02.416774 1435 hybrid_clock.cc:648] HybridClock initialized: now 1786265702416762 us; error 28 us; skew 500 ppm
Aug 09 08:55:02 dist-test-slave-t3q3 krb5kdc[862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265702, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:55:02.419476 1435 init.cc:377] Logged in from keytab as kudu/127.8.154.3@KRBTEST.COM (short username kudu)
I20260809 08:55:02.419868 1435 webserver.cc:460] Webserver started at http://127.8.154.3:33843/ using document root <none> and password file <none>
I20260809 08:55:02.420153 1435 fs_manager.cc:359] Metadata directory not provided
I20260809 08:55:02.420226 1435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:55:02.420385 1435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:55:02.421532 1435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/data/instance:
uuid: "52ae6da16ff7455aa1d0a0b491088b02"
format_stamp: "Formatted at 2026-08-09 08:55:02 on dist-test-slave-t3q3"
server_key: "f9f4dc8a50c7655cef9f0bba176e18b2"
server_key_iv: "5d6db3609d5452828c875d8047f31045"
server_key_version: "encryptionkey@0"
I20260809 08:55:02.421922 1435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance:
uuid: "52ae6da16ff7455aa1d0a0b491088b02"
format_stamp: "Formatted at 2026-08-09 08:55:02 on dist-test-slave-t3q3"
server_key: "f9f4dc8a50c7655cef9f0bba176e18b2"
server_key_iv: "5d6db3609d5452828c875d8047f31045"
server_key_version: "encryptionkey@0"
I20260809 08:55:02.423353 1435 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:55:02.424289 1463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:55:02.424502 1435 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:55:02.424628 1435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/wal
uuid: "52ae6da16ff7455aa1d0a0b491088b02"
format_stamp: "Formatted at 2026-08-09 08:55:02 on dist-test-slave-t3q3"
server_key: "f9f4dc8a50c7655cef9f0bba176e18b2"
server_key_iv: "5d6db3609d5452828c875d8047f31045"
server_key_version: "encryptionkey@0"
I20260809 08:55:02.424715 1435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:55:02.440575 1435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:55:02.442610 1435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:55:02.442785 1435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:55:02.443065 1435 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:55:02.443406 1435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:55:02.443463 1435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:55:02.443506 1435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:55:02.443535 1435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:55:02.450448 1435 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.3:39667
I20260809 08:55:02.450503 1629 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.3:39667 every 8 connection(s)
I20260809 08:55:02.450832 1435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
Aug 09 08:55:02 dist-test-slave-t3q3 krb5kdc[862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265702, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:55:02.458911 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 1435
I20260809 08:55:02.459072 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance
I20260809 08:55:02.459275 8808 external_mini_cluster.cc:1428] Setting key d3def6a07aed4f76c5b521903d443298
I20260809 08:55:02.460572 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:02.451516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:46643 (local address 127.8.154.62:46845)
0809 08:55:02.451589 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 08:55:02.451591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:55:02.451844 (+ 253us) server_negotiation.cc:407] Connection header received
0809 08:55:02.452422 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:55:02.452424 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:55:02.452435 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:55:02.452505 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:55:02.453533 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.454051 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.455090 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.455289 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.455357 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:02.457692 (+ 2335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:55:02.457694 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:55:02.457694 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:55:02.457701 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:55:02.458959 (+ 1258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:55:02.459513 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:55:02.459515 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:55:02.459516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:55:02.459589 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:55:02.459883 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:55:02.459885 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:55:02.459885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:55:02.460006 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:55:02.460106 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 08:55:02.460396 (+ 290us) server_negotiation.cc:299] Negotiation successful
0809 08:55:02.460480 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 08:55:02.460759 1634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:02.451502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.3:46643)
0809 08:55:02.451690 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:02.451727 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 08:55:02.452329 (+ 602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:55:02.452568 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:55:02.452572 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:55:02.453218 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:55:02.453430 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:55:02.453434 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.454144 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.454145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:55:02.454991 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:55:02.454993 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.455409 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.455411 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:55:02.455526 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:02.456159 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:55:02.456169 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:55:02.457598 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:55:02.459052 (+ 1454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:55:02.459061 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:55:02.459063 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:55:02.459387 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:55:02.459663 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:55:02.459664 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:55:02.459664 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:55:02.459800 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:55:02.460097 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:55:02.460105 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:55:02.460255 (+ 150us) client_negotiation.cc:769] Sending connection context
0809 08:55:02.460371 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 08:55:02.460529 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":111,"threads_started":1}
I20260809 08:55:02.461288 1630 heartbeater.cc:347] Connected to a master server at 127.8.154.62:46845
I20260809 08:55:02.461371 1630 heartbeater.cc:464] Registering TS with master...
I20260809 08:55:02.461509 1630 heartbeater.cc:511] Master 127.8.154.62:46845 requested a full tablet report, sending...
I20260809 08:55:02.461841 935 ts_manager.cc:194] Registered new tserver with Master: 52ae6da16ff7455aa1d0a0b491088b02 (127.8.154.3:39667)
I20260809 08:55:02.462184 935 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.3@KRBTEST.COM'} at 127.8.154.3:46643
I20260809 08:55:02.471068 8808 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:55:02.478288 1014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:02.472437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37654 (local address 127.8.154.62:46845)
0809 08:55:02.472516 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 08:55:02.472518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:55:02.472614 (+ 96us) server_negotiation.cc:407] Connection header received
0809 08:55:02.472704 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:55:02.472705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:55:02.472715 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:55:02.472772 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:55:02.473349 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.473831 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.474574 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.474773 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.474847 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:02.475936 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:55:02.475937 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:55:02.475937 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:55:02.475944 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:55:02.477112 (+ 1168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:55:02.477457 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:55:02.477458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:55:02.477458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:55:02.477519 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:55:02.477740 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:55:02.477741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:55:02.477742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:55:02.477885 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:55:02.477991 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 08:55:02.478109 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 08:55:02.478190 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 08:55:02.479813 935 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37654:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:55:02.481436 935 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:55:02.488137 1656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:02.484165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44558 (local address 127.8.154.2:43319)
0809 08:55:02.484327 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 08:55:02.484332 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:55:02.484344 (+ 12us) server_negotiation.cc:407] Connection header received
0809 08:55:02.484388 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:55:02.484390 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:55:02.484407 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:55:02.484710 (+ 303us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:55:02.484790 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.485928 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.487174 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.487690 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.487786 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:02.487853 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 08:55:02.487988 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 08:55:02.488030 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":94,"threads_started":1}
I20260809 08:55:02.488130 1650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:02.483826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:43319 (local address 127.0.0.1:44558)
0809 08:55:02.484126 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:02.484137 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:55:02.484194 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:55:02.484543 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:55:02.484546 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:55:02.484549 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:55:02.484617 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:55:02.484619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.486058 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.486059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:55:02.487106 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:55:02.487108 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.487787 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.487788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:55:02.487871 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:02.487885 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 08:55:02.487960 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 08:55:02.488019 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":89,"threads_started":1}
I20260809 08:55:02.488255 1652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:02.483827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:45617 (local address 127.0.0.1:56032)
0809 08:55:02.484272 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:02.484279 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 08:55:02.484306 (+ 27us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:55:02.484797 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:55:02.484798 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:55:02.484800 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:55:02.484833 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:55:02.484834 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.486089 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.486089 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:55:02.486956 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:55:02.486959 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.487949 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.487950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:55:02.488023 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:02.488028 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:55:02.488115 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 08:55:02.488165 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"mutex_wait_us":7,"spinlock_wait_cycles":384,"thread_start_us":96,"threads_started":1}
I20260809 08:55:02.488548 1654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:02.483900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37838 (local address 127.8.154.3:39667)
0809 08:55:02.484445 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0809 08:55:02.484450 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:55:02.484462 (+ 12us) server_negotiation.cc:407] Connection header received
0809 08:55:02.484501 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:55:02.484504 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:55:02.484521 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:55:02.484597 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:55:02.484825 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.485539 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.487101 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.487779 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.487852 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:02.487908 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 08:55:02.488422 (+ 514us) server_negotiation.cc:299] Negotiation successful
0809 08:55:02.488470 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":84,"threads_started":1}
I20260809 08:55:02.488572 1657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:02.484333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56032 (local address 127.8.154.1:45617)
0809 08:55:02.484551 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 08:55:02.484555 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:55:02.484651 (+ 96us) server_negotiation.cc:407] Connection header received
0809 08:55:02.484695 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:55:02.484698 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:55:02.484713 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:55:02.484786 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:55:02.484895 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.485963 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.487045 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.487779 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.487852 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:02.487907 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 08:55:02.488441 (+ 534us) server_negotiation.cc:299] Negotiation successful
0809 08:55:02.488483 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":82,"threads_started":1}
I20260809 08:55:02.488771 1651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:02.483812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:39667 (local address 127.0.0.1:37838)
0809 08:55:02.483971 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:02.484011 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 08:55:02.484107 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:55:02.484682 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:55:02.484683 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:55:02.484685 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:55:02.484743 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:55:02.484745 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.485656 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.485657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:55:02.487035 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:55:02.487037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.487947 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.487949 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:55:02.488032 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:02.488035 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:55:02.488142 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 08:55:02.488173 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":46,"threads_started":1}
I20260809 08:55:02.488950 1335 tablet_service.cc:1467] Processing CreateTablet for tablet 662f0888fd3c4603b9d826d65f891eb3 (DEFAULT_TABLE table=test-table [id=ed644b5be9cf464fbf4a1c0014737c62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:55:02.489359 1335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 662f0888fd3c4603b9d826d65f891eb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:55:02.489553 1129 tablet_service.cc:1467] Processing CreateTablet for tablet 662f0888fd3c4603b9d826d65f891eb3 (DEFAULT_TABLE table=test-table [id=ed644b5be9cf464fbf4a1c0014737c62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:55:02.489866 1129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 662f0888fd3c4603b9d826d65f891eb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:55:02.492311 1658 tablet_bootstrap.cc:492] T 662f0888fd3c4603b9d826d65f891eb3 P d040e1f52a2743eaaccaa5e30c679477: Bootstrap starting.
I20260809 08:55:02.492626 1660 tablet_bootstrap.cc:492] T 662f0888fd3c4603b9d826d65f891eb3 P c17f5ac68cd74824828707b8ac593ffc: Bootstrap starting.
I20260809 08:55:02.492925 1530 tablet_service.cc:1467] Processing CreateTablet for tablet 662f0888fd3c4603b9d826d65f891eb3 (DEFAULT_TABLE table=test-table [id=ed644b5be9cf464fbf4a1c0014737c62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:55:02.493249 1530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 662f0888fd3c4603b9d826d65f891eb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:55:02.493314 1660 tablet_bootstrap.cc:654] T 662f0888fd3c4603b9d826d65f891eb3 P c17f5ac68cd74824828707b8ac593ffc: Neither blocks nor log segments found. Creating new log.
I20260809 08:55:02.493806 1660 log.cc:824] T 662f0888fd3c4603b9d826d65f891eb3 P c17f5ac68cd74824828707b8ac593ffc: Log is configured to *not* fsync() on all Append() calls
I20260809 08:55:02.494549 1660 tablet_bootstrap.cc:492] T 662f0888fd3c4603b9d826d65f891eb3 P c17f5ac68cd74824828707b8ac593ffc: No bootstrap required, opened a new log
I20260809 08:55:02.494634 1660 ts_tablet_manager.cc:1397] T 662f0888fd3c4603b9d826d65f891eb3 P c17f5ac68cd74824828707b8ac593ffc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:55:02.495925 1664 tablet_bootstrap.cc:492] T 662f0888fd3c4603b9d826d65f891eb3 P 52ae6da16ff7455aa1d0a0b491088b02: Bootstrap starting.
I20260809 08:55:02.496497 1658 tablet_bootstrap.cc:654] T 662f0888fd3c4603b9d826d65f891eb3 P d040e1f52a2743eaaccaa5e30c679477: Neither blocks nor log segments found. Creating new log.
I20260809 08:55:02.496482 1660 raft_consensus.cc:348] T 662f0888fd3c4603b9d826d65f891eb3 P c17f5ac68cd74824828707b8ac593ffc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ae6da16ff7455aa1d0a0b491088b02" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 39667 } } peers { permanent_uuid: "c17f5ac68cd74824828707b8ac593ffc" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45617 } } peers { permanent_uuid: "d040e1f52a2743eaaccaa5e30c679477" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43319 } }
I20260809 08:55:02.496657 1664 tablet_bootstrap.cc:654] T 662f0888fd3c4603b9d826d65f891eb3 P 52ae6da16ff7455aa1d0a0b491088b02: Neither blocks nor log segments found. Creating new log.
I20260809 08:55:02.496701 1660 raft_consensus.cc:374] T 662f0888fd3c4603b9d826d65f891eb3 P c17f5ac68cd74824828707b8ac593ffc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:55:02.496752 1660 raft_consensus.cc:729] T 662f0888fd3c4603b9d826d65f891eb3 P c17f5ac68cd74824828707b8ac593ffc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c17f5ac68cd74824828707b8ac593ffc, State: Initialized, Role: FOLLOWER
I20260809 08:55:02.496886 1660 consensus_queue.cc:260] T 662f0888fd3c4603b9d826d65f891eb3 P c17f5ac68cd74824828707b8ac593ffc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ae6da16ff7455aa1d0a0b491088b02" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 39667 } } peers { permanent_uuid: "c17f5ac68cd74824828707b8ac593ffc" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45617 } } peers { permanent_uuid: "d040e1f52a2743eaaccaa5e30c679477" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43319 } }
I20260809 08:55:02.497115 1658 log.cc:824] T 662f0888fd3c4603b9d826d65f891eb3 P d040e1f52a2743eaaccaa5e30c679477: Log is configured to *not* fsync() on all Append() calls
I20260809 08:55:02.497226 1660 ts_tablet_manager.cc:1428] T 662f0888fd3c4603b9d826d65f891eb3 P c17f5ac68cd74824828707b8ac593ffc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:55:02.497272 1231 heartbeater.cc:503] Master 127.8.154.62:46845 was elected leader, sending a full tablet report...
I20260809 08:55:02.497308 1664 log.cc:824] T 662f0888fd3c4603b9d826d65f891eb3 P 52ae6da16ff7455aa1d0a0b491088b02: Log is configured to *not* fsync() on all Append() calls
I20260809 08:55:02.497888 1658 tablet_bootstrap.cc:492] T 662f0888fd3c4603b9d826d65f891eb3 P d040e1f52a2743eaaccaa5e30c679477: No bootstrap required, opened a new log
I20260809 08:55:02.497978 1658 ts_tablet_manager.cc:1397] T 662f0888fd3c4603b9d826d65f891eb3 P d040e1f52a2743eaaccaa5e30c679477: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260809 08:55:02.498183 1664 tablet_bootstrap.cc:492] T 662f0888fd3c4603b9d826d65f891eb3 P 52ae6da16ff7455aa1d0a0b491088b02: No bootstrap required, opened a new log
I20260809 08:55:02.498236 1664 ts_tablet_manager.cc:1397] T 662f0888fd3c4603b9d826d65f891eb3 P 52ae6da16ff7455aa1d0a0b491088b02: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 08:55:02.499616 1664 raft_consensus.cc:348] T 662f0888fd3c4603b9d826d65f891eb3 P 52ae6da16ff7455aa1d0a0b491088b02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ae6da16ff7455aa1d0a0b491088b02" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 39667 } } peers { permanent_uuid: "c17f5ac68cd74824828707b8ac593ffc" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45617 } } peers { permanent_uuid: "d040e1f52a2743eaaccaa5e30c679477" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43319 } }
I20260809 08:55:02.499637 1658 raft_consensus.cc:348] T 662f0888fd3c4603b9d826d65f891eb3 P d040e1f52a2743eaaccaa5e30c679477 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ae6da16ff7455aa1d0a0b491088b02" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 39667 } } peers { permanent_uuid: "c17f5ac68cd74824828707b8ac593ffc" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45617 } } peers { permanent_uuid: "d040e1f52a2743eaaccaa5e30c679477" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43319 } }
I20260809 08:55:02.499810 1664 raft_consensus.cc:374] T 662f0888fd3c4603b9d826d65f891eb3 P 52ae6da16ff7455aa1d0a0b491088b02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:55:02.499821 1658 raft_consensus.cc:374] T 662f0888fd3c4603b9d826d65f891eb3 P d040e1f52a2743eaaccaa5e30c679477 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:55:02.499868 1658 raft_consensus.cc:729] T 662f0888fd3c4603b9d826d65f891eb3 P d040e1f52a2743eaaccaa5e30c679477 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d040e1f52a2743eaaccaa5e30c679477, State: Initialized, Role: FOLLOWER
I20260809 08:55:02.499872 1664 raft_consensus.cc:729] T 662f0888fd3c4603b9d826d65f891eb3 P 52ae6da16ff7455aa1d0a0b491088b02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ae6da16ff7455aa1d0a0b491088b02, State: Initialized, Role: FOLLOWER
I20260809 08:55:02.499992 1658 consensus_queue.cc:260] T 662f0888fd3c4603b9d826d65f891eb3 P d040e1f52a2743eaaccaa5e30c679477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ae6da16ff7455aa1d0a0b491088b02" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 39667 } } peers { permanent_uuid: "c17f5ac68cd74824828707b8ac593ffc" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45617 } } peers { permanent_uuid: "d040e1f52a2743eaaccaa5e30c679477" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43319 } }
I20260809 08:55:02.500015 1664 consensus_queue.cc:260] T 662f0888fd3c4603b9d826d65f891eb3 P 52ae6da16ff7455aa1d0a0b491088b02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ae6da16ff7455aa1d0a0b491088b02" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 39667 } } peers { permanent_uuid: "c17f5ac68cd74824828707b8ac593ffc" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45617 } } peers { permanent_uuid: "d040e1f52a2743eaaccaa5e30c679477" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43319 } }
I20260809 08:55:02.500349 1658 ts_tablet_manager.cc:1428] T 662f0888fd3c4603b9d826d65f891eb3 P d040e1f52a2743eaaccaa5e30c679477: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:55:02.500363 1429 heartbeater.cc:503] Master 127.8.154.62:46845 was elected leader, sending a full tablet report...
I20260809 08:55:02.500344 1664 ts_tablet_manager.cc:1428] T 662f0888fd3c4603b9d826d65f891eb3 P 52ae6da16ff7455aa1d0a0b491088b02: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:55:02.500411 1630 heartbeater.cc:503] Master 127.8.154.62:46845 was elected leader, sending a full tablet report...
W20260809 08:55:02.536573 1431 tablet.cc:2369] T 662f0888fd3c4603b9d826d65f891eb3 P d040e1f52a2743eaaccaa5e30c679477: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 08:55:02.624516 1233 tablet.cc:2369] T 662f0888fd3c4603b9d826d65f891eb3 P c17f5ac68cd74824828707b8ac593ffc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:55:02.697824 1673 raft_consensus.cc:482] T 662f0888fd3c4603b9d826d65f891eb3 P d040e1f52a2743eaaccaa5e30c679477 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:55:02.698026 1673 raft_consensus.cc:504] T 662f0888fd3c4603b9d826d65f891eb3 P d040e1f52a2743eaaccaa5e30c679477 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ae6da16ff7455aa1d0a0b491088b02" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 39667 } } peers { permanent_uuid: "c17f5ac68cd74824828707b8ac593ffc" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45617 } } peers { permanent_uuid: "d040e1f52a2743eaaccaa5e30c679477" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43319 } }
I20260809 08:55:02.698487 1673 leader_election.cc:288] T 662f0888fd3c4603b9d826d65f891eb3 P d040e1f52a2743eaaccaa5e30c679477 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52ae6da16ff7455aa1d0a0b491088b02 (127.8.154.3:39667), c17f5ac68cd74824828707b8ac593ffc (127.8.154.1:45617)
W20260809 08:55:02.701386 1632 tablet.cc:2369] T 662f0888fd3c4603b9d826d65f891eb3 P 52ae6da16ff7455aa1d0a0b491088b02: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:55:02.701472 1432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:02.698692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:39667 (local address 127.8.154.2:60121)
0809 08:55:02.698753 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:02.698765 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:55:02.698815 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:55:02.699009 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:55:02.699010 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:55:02.699012 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:55:02.699093 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:55:02.699095 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.699756 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.699757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:55:02.700578 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:55:02.700581 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.701161 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.701162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:55:02.701290 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:02.701294 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:55:02.701351 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 08:55:02.701392 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 08:55:02.701530 1654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:02.698746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:60121 (local address 127.8.154.3:39667)
0809 08:55:02.698791 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 08:55:02.698793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:55:02.698803 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:55:02.698897 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:55:02.698899 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:55:02.698921 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:55:02.699004 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:55:02.699185 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.699666 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.700686 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.701077 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.701170 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:02.701183 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 08:55:02.701377 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 08:55:02.701426 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 08:55:02.701613 1657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:02.698769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:50739 (local address 127.8.154.1:45617)
0809 08:55:02.698833 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 08:55:02.698835 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:55:02.698971 (+ 136us) server_negotiation.cc:407] Connection header received
0809 08:55:02.699101 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:55:02.699103 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:55:02.699116 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:55:02.699181 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:55:02.699334 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.699804 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.700870 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.701234 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.701316 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:02.701326 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 08:55:02.701493 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 08:55:02.701533 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 08:55:02.701753 1665 raft_consensus.cc:482] T 662f0888fd3c4603b9d826d65f891eb3 P c17f5ac68cd74824828707b8ac593ffc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:55:02.701821 1676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:02.698694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:45617 (local address 127.8.154.2:50739)
0809 08:55:02.698889 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:02.698899 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:55:02.698992 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:55:02.699185 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:55:02.699186 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:55:02.699187 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:55:02.699262 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:55:02.699264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.699890 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.699891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:55:02.700781 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:55:02.700784 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.701330 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.701331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:55:02.701405 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:02.701409 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:55:02.701690 (+ 281us) client_negotiation.cc:240] Negotiation successful
0809 08:55:02.701736 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":12,"thread_start_us":115,"threads_started":1}
I20260809 08:55:02.701841 1665 raft_consensus.cc:504] T 662f0888fd3c4603b9d826d65f891eb3 P c17f5ac68cd74824828707b8ac593ffc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ae6da16ff7455aa1d0a0b491088b02" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 39667 } } peers { permanent_uuid: "c17f5ac68cd74824828707b8ac593ffc" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45617 } } peers { permanent_uuid: "d040e1f52a2743eaaccaa5e30c679477" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43319 } }
I20260809 08:55:02.702075 1565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "662f0888fd3c4603b9d826d65f891eb3" candidate_uuid: "d040e1f52a2743eaaccaa5e30c679477" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ae6da16ff7455aa1d0a0b491088b02" is_pre_election: true
I20260809 08:55:02.702209 1565 raft_consensus.cc:2454] T 662f0888fd3c4603b9d826d65f891eb3 P 52ae6da16ff7455aa1d0a0b491088b02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d040e1f52a2743eaaccaa5e30c679477 in term 0.
I20260809 08:55:02.702186 1161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "662f0888fd3c4603b9d826d65f891eb3" candidate_uuid: "d040e1f52a2743eaaccaa5e30c679477" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c17f5ac68cd74824828707b8ac593ffc" is_pre_election: true
I20260809 08:55:02.702200 1665 leader_election.cc:288] T 662f0888fd3c4603b9d826d65f891eb3 P c17f5ac68cd74824828707b8ac593ffc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52ae6da16ff7455aa1d0a0b491088b02 (127.8.154.3:39667), d040e1f52a2743eaaccaa5e30c679477 (127.8.154.2:43319)
I20260809 08:55:02.702304 1161 raft_consensus.cc:2454] T 662f0888fd3c4603b9d826d65f891eb3 P c17f5ac68cd74824828707b8ac593ffc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d040e1f52a2743eaaccaa5e30c679477 in term 0.
I20260809 08:55:02.702529 1265 leader_election.cc:302] T 662f0888fd3c4603b9d826d65f891eb3 P d040e1f52a2743eaaccaa5e30c679477 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ae6da16ff7455aa1d0a0b491088b02, d040e1f52a2743eaaccaa5e30c679477; no voters:
I20260809 08:55:02.702688 1673 raft_consensus.cc:2781] T 662f0888fd3c4603b9d826d65f891eb3 P d040e1f52a2743eaaccaa5e30c679477 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:55:02.702746 1673 raft_consensus.cc:482] T 662f0888fd3c4603b9d826d65f891eb3 P d040e1f52a2743eaaccaa5e30c679477 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:55:02.702776 1673 raft_consensus.cc:3037] T 662f0888fd3c4603b9d826d65f891eb3 P d040e1f52a2743eaaccaa5e30c679477 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:55:02.703778 1673 raft_consensus.cc:504] T 662f0888fd3c4603b9d826d65f891eb3 P d040e1f52a2743eaaccaa5e30c679477 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ae6da16ff7455aa1d0a0b491088b02" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 39667 } } peers { permanent_uuid: "c17f5ac68cd74824828707b8ac593ffc" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45617 } } peers { permanent_uuid: "d040e1f52a2743eaaccaa5e30c679477" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43319 } }
I20260809 08:55:02.703926 1673 leader_election.cc:288] T 662f0888fd3c4603b9d826d65f891eb3 P d040e1f52a2743eaaccaa5e30c679477 [CANDIDATE]: Term 1 election: Requested vote from peers 52ae6da16ff7455aa1d0a0b491088b02 (127.8.154.3:39667), c17f5ac68cd74824828707b8ac593ffc (127.8.154.1:45617)
I20260809 08:55:02.704118 1161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "662f0888fd3c4603b9d826d65f891eb3" candidate_uuid: "d040e1f52a2743eaaccaa5e30c679477" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c17f5ac68cd74824828707b8ac593ffc"
I20260809 08:55:02.704205 1161 raft_consensus.cc:3037] T 662f0888fd3c4603b9d826d65f891eb3 P c17f5ac68cd74824828707b8ac593ffc [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:55:02.704185 1565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "662f0888fd3c4603b9d826d65f891eb3" candidate_uuid: "d040e1f52a2743eaaccaa5e30c679477" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ae6da16ff7455aa1d0a0b491088b02"
I20260809 08:55:02.704274 1565 raft_consensus.cc:3037] T 662f0888fd3c4603b9d826d65f891eb3 P 52ae6da16ff7455aa1d0a0b491088b02 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:55:02.705051 1161 raft_consensus.cc:2454] T 662f0888fd3c4603b9d826d65f891eb3 P c17f5ac68cd74824828707b8ac593ffc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d040e1f52a2743eaaccaa5e30c679477 in term 1.
I20260809 08:55:02.705209 1565 raft_consensus.cc:2454] T 662f0888fd3c4603b9d826d65f891eb3 P 52ae6da16ff7455aa1d0a0b491088b02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d040e1f52a2743eaaccaa5e30c679477 in term 1.
I20260809 08:55:02.705317 1264 leader_election.cc:302] T 662f0888fd3c4603b9d826d65f891eb3 P d040e1f52a2743eaaccaa5e30c679477 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c17f5ac68cd74824828707b8ac593ffc, d040e1f52a2743eaaccaa5e30c679477; no voters:
I20260809 08:55:02.705989 1673 raft_consensus.cc:2781] T 662f0888fd3c4603b9d826d65f891eb3 P d040e1f52a2743eaaccaa5e30c679477 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:55:02.706187 1673 raft_consensus.cc:686] T 662f0888fd3c4603b9d826d65f891eb3 P d040e1f52a2743eaaccaa5e30c679477 [term 1 LEADER]: Becoming Leader. State: Replica: d040e1f52a2743eaaccaa5e30c679477, State: Running, Role: LEADER
I20260809 08:55:02.706278 1654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:02.702433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:50963 (local address 127.8.154.3:39667)
0809 08:55:02.702612 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 08:55:02.702614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:55:02.702767 (+ 153us) server_negotiation.cc:407] Connection header received
0809 08:55:02.702791 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:55:02.702791 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:55:02.702800 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:55:02.702852 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:55:02.703060 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.703694 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.705326 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.705883 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.705923 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:02.705930 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 08:55:02.706177 (+ 247us) server_negotiation.cc:299] Negotiation successful
0809 08:55:02.706222 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156}
I20260809 08:55:02.706269 1656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:02.702707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:60607 (local address 127.8.154.2:43319)
0809 08:55:02.702744 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 08:55:02.702746 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:55:02.702882 (+ 136us) server_negotiation.cc:407] Connection header received
0809 08:55:02.702992 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:55:02.702993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:55:02.703004 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:55:02.703055 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:55:02.703226 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.703694 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.705165 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.705813 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.705892 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:02.705902 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 08:55:02.706167 (+ 265us) server_negotiation.cc:299] Negotiation successful
0809 08:55:02.706207 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:55:02.706290 1677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:02.702474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:43319 (local address 127.8.154.1:60607)
0809 08:55:02.702807 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:02.702819 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:55:02.702914 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:55:02.703103 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:55:02.703104 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:55:02.703105 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:55:02.703156 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:55:02.703156 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.703787 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.703788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:55:02.705042 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:55:02.705046 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.705933 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.705934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:55:02.706013 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:02.706017 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:55:02.706159 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 08:55:02.706205 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":14,"thread_start_us":105,"threads_started":1}
I20260809 08:55:02.706308 1673 consensus_queue.cc:237] T 662f0888fd3c4603b9d826d65f891eb3 P d040e1f52a2743eaaccaa5e30c679477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ae6da16ff7455aa1d0a0b491088b02" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 39667 } } peers { permanent_uuid: "c17f5ac68cd74824828707b8ac593ffc" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45617 } } peers { permanent_uuid: "d040e1f52a2743eaaccaa5e30c679477" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43319 } }
I20260809 08:55:02.706557 1678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:02.702474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:39667 (local address 127.8.154.1:50963)
0809 08:55:02.702683 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:02.702692 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:55:02.702744 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:55:02.702855 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:55:02.702856 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:55:02.702857 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:55:02.702923 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:55:02.702924 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.703799 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.703800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:55:02.705189 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:55:02.705192 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.705990 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.705991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:55:02.706078 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:02.706084 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 08:55:02.706407 (+ 323us) client_negotiation.cc:240] Negotiation successful
0809 08:55:02.706453 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":184,"thread_start_us":88,"threads_started":1}
I20260809 08:55:02.706606 1371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "662f0888fd3c4603b9d826d65f891eb3" candidate_uuid: "c17f5ac68cd74824828707b8ac593ffc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d040e1f52a2743eaaccaa5e30c679477" is_pre_election: true
I20260809 08:55:02.706883 1565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "662f0888fd3c4603b9d826d65f891eb3" candidate_uuid: "c17f5ac68cd74824828707b8ac593ffc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ae6da16ff7455aa1d0a0b491088b02" is_pre_election: true
I20260809 08:55:02.706979 1565 raft_consensus.cc:2379] T 662f0888fd3c4603b9d826d65f891eb3 P 52ae6da16ff7455aa1d0a0b491088b02 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c17f5ac68cd74824828707b8ac593ffc in current term 1: Already voted for candidate d040e1f52a2743eaaccaa5e30c679477 in this term.
I20260809 08:55:02.707240 1059 leader_election.cc:302] T 662f0888fd3c4603b9d826d65f891eb3 P c17f5ac68cd74824828707b8ac593ffc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c17f5ac68cd74824828707b8ac593ffc; no voters: 52ae6da16ff7455aa1d0a0b491088b02, d040e1f52a2743eaaccaa5e30c679477
I20260809 08:55:02.707284 935 catalog_manager.cc:5495] T 662f0888fd3c4603b9d826d65f891eb3 P d040e1f52a2743eaaccaa5e30c679477 reported cstate change: term changed from 0 to 1, leader changed from <none> to d040e1f52a2743eaaccaa5e30c679477 (127.8.154.2). New cstate: current_term: 1 leader_uuid: "d040e1f52a2743eaaccaa5e30c679477" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ae6da16ff7455aa1d0a0b491088b02" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 39667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c17f5ac68cd74824828707b8ac593ffc" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d040e1f52a2743eaaccaa5e30c679477" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43319 } health_report { overall_health: HEALTHY } } }
I20260809 08:55:02.707415 1665 raft_consensus.cc:2726] T 662f0888fd3c4603b9d826d65f891eb3 P c17f5ac68cd74824828707b8ac593ffc [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 08:55:02.714288 1656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:02.712110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44564 (local address 127.8.154.2:43319)
0809 08:55:02.712156 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 08:55:02.712158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:55:02.712167 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:55:02.712224 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:55:02.712225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:55:02.712240 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:55:02.712300 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:55:02.712419 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.712858 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.713621 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:02.713803 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:02.713868 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:02.714010 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:55:02.714102 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:55:02.714165 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 08:55:02.714201 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 08:55:02.714218 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 08:55:02.715955 1161 raft_consensus.cc:1261] T 662f0888fd3c4603b9d826d65f891eb3 P c17f5ac68cd74824828707b8ac593ffc [term 1 FOLLOWER]: Refusing update from remote peer d040e1f52a2743eaaccaa5e30c679477: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:55:02.715955 1565 raft_consensus.cc:1261] T 662f0888fd3c4603b9d826d65f891eb3 P 52ae6da16ff7455aa1d0a0b491088b02 [term 1 FOLLOWER]: Refusing update from remote peer d040e1f52a2743eaaccaa5e30c679477: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:55:02.716312 1673 consensus_queue.cc:1035] T 662f0888fd3c4603b9d826d65f891eb3 P d040e1f52a2743eaaccaa5e30c679477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c17f5ac68cd74824828707b8ac593ffc" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:55:02.716457 1679 consensus_queue.cc:1035] T 662f0888fd3c4603b9d826d65f891eb3 P d040e1f52a2743eaaccaa5e30c679477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ae6da16ff7455aa1d0a0b491088b02" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 39667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:55:02.721107 935 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:55:02.721236 935 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:55:02.721975 935 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10: Sending DeleteTablet for 3 replicas of tablet 662f0888fd3c4603b9d826d65f891eb3
I20260809 08:55:02.722308 1530 tablet_service.cc:1514] Processing DeleteTablet for tablet 662f0888fd3c4603b9d826d65f891eb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:55:02 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:37838
I20260809 08:55:02.722307 1129 tablet_service.cc:1514] Processing DeleteTablet for tablet 662f0888fd3c4603b9d826d65f891eb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:55:02 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:56032
I20260809 08:55:02.722550 1696 tablet_replica.cc:330] T 662f0888fd3c4603b9d826d65f891eb3 P c17f5ac68cd74824828707b8ac593ffc: stopping tablet replica
I20260809 08:55:02.722523 1335 tablet_service.cc:1514] Processing DeleteTablet for tablet 662f0888fd3c4603b9d826d65f891eb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:55:02 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:44558
I20260809 08:55:02.722674 1696 raft_consensus.cc:2229] T 662f0888fd3c4603b9d826d65f891eb3 P c17f5ac68cd74824828707b8ac593ffc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:55:02.722756 1697 tablet_replica.cc:330] T 662f0888fd3c4603b9d826d65f891eb3 P d040e1f52a2743eaaccaa5e30c679477: stopping tablet replica
I20260809 08:55:02.722761 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 1026
I20260809 08:55:02.722857 1696 raft_consensus.cc:2258] T 662f0888fd3c4603b9d826d65f891eb3 P c17f5ac68cd74824828707b8ac593ffc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:55:02.722892 1697 raft_consensus.cc:2229] T 662f0888fd3c4603b9d826d65f891eb3 P d040e1f52a2743eaaccaa5e30c679477 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:55:02.723018 1697 raft_consensus.cc:2258] T 662f0888fd3c4603b9d826d65f891eb3 P d040e1f52a2743eaaccaa5e30c679477 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:55:02.734278 1697 ts_tablet_manager.cc:1905] T 662f0888fd3c4603b9d826d65f891eb3 P d040e1f52a2743eaaccaa5e30c679477: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:55:02.736174 1697 ts_tablet_manager.cc:1918] T 662f0888fd3c4603b9d826d65f891eb3 P d040e1f52a2743eaaccaa5e30c679477: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:55:02.736292 1697 log.cc:1196] T 662f0888fd3c4603b9d826d65f891eb3 P d040e1f52a2743eaaccaa5e30c679477: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/wal/wals/662f0888fd3c4603b9d826d65f891eb3
I20260809 08:55:02.736630 1697 ts_tablet_manager.cc:1939] T 662f0888fd3c4603b9d826d65f891eb3 P d040e1f52a2743eaaccaa5e30c679477: Deleting consensus metadata
I20260809 08:55:02.737217 902 catalog_manager.cc:4841] TS d040e1f52a2743eaaccaa5e30c679477 (127.8.154.2:43319): tablet 662f0888fd3c4603b9d826d65f891eb3 (table test-table [id=ed644b5be9cf464fbf4a1c0014737c62]) successfully deleted
I20260809 08:55:02.740284 1695 tablet_replica.cc:330] T 662f0888fd3c4603b9d826d65f891eb3 P 52ae6da16ff7455aa1d0a0b491088b02: stopping tablet replica
I20260809 08:55:02.740423 1695 raft_consensus.cc:2229] T 662f0888fd3c4603b9d826d65f891eb3 P 52ae6da16ff7455aa1d0a0b491088b02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:55:02.740543 1695 raft_consensus.cc:2258] T 662f0888fd3c4603b9d826d65f891eb3 P 52ae6da16ff7455aa1d0a0b491088b02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:55:02.742455 1695 ts_tablet_manager.cc:1905] T 662f0888fd3c4603b9d826d65f891eb3 P 52ae6da16ff7455aa1d0a0b491088b02: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 08:55:02.742518 904 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.154.1:45617 (error 108)
I20260809 08:55:02.742735 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 1239
I20260809 08:55:02.743144 1651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:02.743019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:45617 (local address 127.0.0.1:56038)
0809 08:55:02.743069 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:02.743101 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.1:45617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 08:55:02.743378 904 catalog_manager.cc:4584] TS c17f5ac68cd74824828707b8ac593ffc (127.8.154.1:45617): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 662f0888fd3c4603b9d826d65f891eb3: Network error: Client connection negotiation failed: client connection to 127.8.154.1:45617: connect: Connection refused (error 111)
I20260809 08:55:02.745400 1695 ts_tablet_manager.cc:1918] T 662f0888fd3c4603b9d826d65f891eb3 P 52ae6da16ff7455aa1d0a0b491088b02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:55:02.745498 1695 log.cc:1196] T 662f0888fd3c4603b9d826d65f891eb3 P 52ae6da16ff7455aa1d0a0b491088b02: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/wal/wals/662f0888fd3c4603b9d826d65f891eb3
I20260809 08:55:02.745772 1695 ts_tablet_manager.cc:1939] T 662f0888fd3c4603b9d826d65f891eb3 P 52ae6da16ff7455aa1d0a0b491088b02: Deleting consensus metadata
I20260809 08:55:02.746202 905 catalog_manager.cc:4841] TS 52ae6da16ff7455aa1d0a0b491088b02 (127.8.154.3:39667): tablet 662f0888fd3c4603b9d826d65f891eb3 (table test-table [id=ed644b5be9cf464fbf4a1c0014737c62]) successfully deleted
I20260809 08:55:02.752319 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 1435
I20260809 08:55:02.763244 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 879
I20260809 08:55:22.774098 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:46845
--webserver_interface=127.8.154.62
--webserver_port=43855
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:39543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:46845
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:55:22.858114 1701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:55:22.858341 1701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:55:22.858382 1701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:55:22.859493 1701 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 08:55:22.859558 1701 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 08:55:22.859582 1701 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 08:55:22.859612 1701 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:55:22.859640 1701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:55:22.859678 1701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:55:22.859707 1701 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:55:22.859736 1701 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:55:22.861317 1701 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:39543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:46845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:46845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=43855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:55:22.861595 1701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:55:22.861821 1701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:55:22.871157 1710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:55:22.871181 1712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:55:22.871459 1701 server_base.cc:1034] running on GCE node
W20260809 08:55:22.871201 1715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:55:22.871822 1701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:55:22.872241 1701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:55:22.873392 1701 hybrid_clock.cc:648] HybridClock initialized: now 1786265722873378 us; error 32 us; skew 500 ppm
Aug 09 08:55:22 dist-test-slave-t3q3 krb5kdc[862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265722, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:55:22.880899 1701 init.cc:377] Logged in from keytab as kudu/127.8.154.62@KRBTEST.COM (short username kudu)
I20260809 08:55:22.881374 1701 webserver.cc:460] Webserver started at http://127.8.154.62:43855/ using document root <none> and password file <none>
I20260809 08:55:22.881639 1701 fs_manager.cc:359] Metadata directory not provided
I20260809 08:55:22.881727 1701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:55:22.883160 1701 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:55:22.883852 1725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:55:22.884027 1701 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:55:22.884217 1701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "75ab210e8bd44fafb02a3561a90adb10"
format_stamp: "Formatted at 2026-08-09 08:55:01 on dist-test-slave-t3q3"
server_key: "8b389bee1bba5c18ca13141e7ece957f"
server_key_iv: "3b3f285f6511c6ee661d89f023f36d80"
server_key_version: "encryptionkey@0"
I20260809 08:55:22.884642 1701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:55:22.901481 1701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:55:22.903653 1701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:55:22.903836 1701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:55:22.908696 1701 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:46845
I20260809 08:55:22.908728 1809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:46845 every 8 connection(s)
I20260809 08:55:22.909188 1701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:55:22.910418 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 1701
I20260809 08:55:22.910775 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.1:45617
--local_ip_for_outbound_sockets=127.8.154.1
--tserver_master_addrs=127.8.154.62:46845
--webserver_port=43363
--webserver_interface=127.8.154.1
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:39543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:55:22.910790 1811 sys_catalog.cc:263] Verifying existing consensus state
I20260809 08:55:22.911562 1811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10: Bootstrap starting.
I20260809 08:55:22.920697 1811 log.cc:824] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10: Log is configured to *not* fsync() on all Append() calls
I20260809 08:55:22.922812 1811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 08:55:22.923133 1811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10: Bootstrap complete.
I20260809 08:55:22.925892 1811 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75ab210e8bd44fafb02a3561a90adb10" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46845 } }
I20260809 08:55:22.926344 1811 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75ab210e8bd44fafb02a3561a90adb10, State: Initialized, Role: FOLLOWER
I20260809 08:55:22.926527 1811 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75ab210e8bd44fafb02a3561a90adb10" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46845 } }
I20260809 08:55:22.926621 1811 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:55:22.926680 1811 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:55:22.926724 1811 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [term 1 FOLLOWER]: Advancing to term 2
I20260809 08:55:22.927718 1811 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75ab210e8bd44fafb02a3561a90adb10" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46845 } }
I20260809 08:55:22.927803 1811 leader_election.cc:302] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75ab210e8bd44fafb02a3561a90adb10; no voters:
I20260809 08:55:22.928030 1811 leader_election.cc:288] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 08:55:22.928087 1815 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 08:55:22.928212 1815 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [term 2 LEADER]: Becoming Leader. State: Replica: 75ab210e8bd44fafb02a3561a90adb10, State: Running, Role: LEADER
I20260809 08:55:22.928329 1811 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:55:22.928375 1815 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75ab210e8bd44fafb02a3561a90adb10" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46845 } }
I20260809 08:55:22.928761 1817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "75ab210e8bd44fafb02a3561a90adb10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75ab210e8bd44fafb02a3561a90adb10" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46845 } } }
I20260809 08:55:22.928870 1817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [sys.catalog]: This master's current role is: LEADER
I20260809 08:55:22.929194 1827 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:55:22.929199 1819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75ab210e8bd44fafb02a3561a90adb10. Latest consensus state: current_term: 2 leader_uuid: "75ab210e8bd44fafb02a3561a90adb10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75ab210e8bd44fafb02a3561a90adb10" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46845 } } }
I20260809 08:55:22.929417 1819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [sys.catalog]: This master's current role is: LEADER
I20260809 08:55:22.930089 1827 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:55:22.930245 1827 catalog_manager.cc:1259] Loaded cluster ID: 500c1f97dbb24587b8b25ae17323d610
I20260809 08:55:22.930338 1827 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:55:22.931775 1827 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:55:22.932005 1827 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10: Loaded TSK: 0
I20260809 08:55:22.941910 1827 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10: Generated new TSK 1
I20260809 08:55:22.942008 1827 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 08:55:23.003394 1813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:55:23.003639 1813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:55:23.003679 1813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:55:23.004877 1813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:55:23.004944 1813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:55:23.004989 1813 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.1
I20260809 08:55:23.006491 1813 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:39543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.1:45617
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.154.1
--webserver_port=43363
--tserver_master_addrs=127.8.154.62:46845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1813
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.1
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:55:23.006752 1813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:55:23.006982 1813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:55:23.013051 1846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:55:23.013195 1813 server_base.cc:1034] running on GCE node
W20260809 08:55:23.013198 1848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:55:23.013279 1845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:55:23.013563 1813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:55:23.013854 1813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:55:23.015036 1813 hybrid_clock.cc:648] HybridClock initialized: now 1786265723015025 us; error 59 us; skew 500 ppm
Aug 09 08:55:23 dist-test-slave-t3q3 krb5kdc[862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265723, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:55:23.018124 1813 init.cc:377] Logged in from keytab as kudu/127.8.154.1@KRBTEST.COM (short username kudu)
I20260809 08:55:23.018554 1813 webserver.cc:460] Webserver started at http://127.8.154.1:43363/ using document root <none> and password file <none>
I20260809 08:55:23.018783 1813 fs_manager.cc:359] Metadata directory not provided
I20260809 08:55:23.018841 1813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:55:23.020318 1813 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:55:23.021078 1856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:55:23.021270 1813 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:55:23.021386 1813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/wal
uuid: "c17f5ac68cd74824828707b8ac593ffc"
format_stamp: "Formatted at 2026-08-09 08:55:02 on dist-test-slave-t3q3"
server_key: "8a084a3118dae1161263b0b7dfb9ff6a"
server_key_iv: "fe473b1bc4075b391181955ffcea863b"
server_key_version: "encryptionkey@0"
I20260809 08:55:23.021739 1813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:55:23.034875 1813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:55:23.037140 1813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:55:23.037303 1813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:55:23.037555 1813 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:55:23.038010 1868 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 08:55:23.040007 1813 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 08:55:23.040134 1813 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 08:55:23.040198 1813 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 08:55:23.040855 1813 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 08:55:23.040915 1813 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 08:55:23.041177 1868 tablet_bootstrap.cc:492] T 662f0888fd3c4603b9d826d65f891eb3 P c17f5ac68cd74824828707b8ac593ffc: Bootstrap starting.
I20260809 08:55:23.048460 1813 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.1:45617
I20260809 08:55:23.048509 2024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.1:45617 every 8 connection(s)
I20260809 08:55:23.048902 1813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
I20260809 08:55:23.052546 1868 log.cc:824] T 662f0888fd3c4603b9d826d65f891eb3 P c17f5ac68cd74824828707b8ac593ffc: Log is configured to *not* fsync() on all Append() calls
I20260809 08:55:23.054713 1868 tablet_bootstrap.cc:492] T 662f0888fd3c4603b9d826d65f891eb3 P c17f5ac68cd74824828707b8ac593ffc: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 08:55:23.055017 1868 tablet_bootstrap.cc:492] T 662f0888fd3c4603b9d826d65f891eb3 P c17f5ac68cd74824828707b8ac593ffc: Bootstrap complete.
I20260809 08:55:23.056005 1868 ts_tablet_manager.cc:1397] T 662f0888fd3c4603b9d826d65f891eb3 P c17f5ac68cd74824828707b8ac593ffc: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.004s
I20260809 08:55:23.057071 1868 raft_consensus.cc:348] T 662f0888fd3c4603b9d826d65f891eb3 P c17f5ac68cd74824828707b8ac593ffc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ae6da16ff7455aa1d0a0b491088b02" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 39667 } } peers { permanent_uuid: "c17f5ac68cd74824828707b8ac593ffc" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45617 } } peers { permanent_uuid: "d040e1f52a2743eaaccaa5e30c679477" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43319 } }
I20260809 08:55:23.057416 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 1813
I20260809 08:55:23.057422 1868 raft_consensus.cc:729] T 662f0888fd3c4603b9d826d65f891eb3 P c17f5ac68cd74824828707b8ac593ffc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c17f5ac68cd74824828707b8ac593ffc, State: Initialized, Role: FOLLOWER
I20260809 08:55:23.057608 1868 consensus_queue.cc:260] T 662f0888fd3c4603b9d826d65f891eb3 P c17f5ac68cd74824828707b8ac593ffc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ae6da16ff7455aa1d0a0b491088b02" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 39667 } } peers { permanent_uuid: "c17f5ac68cd74824828707b8ac593ffc" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45617 } } peers { permanent_uuid: "d040e1f52a2743eaaccaa5e30c679477" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43319 } }
I20260809 08:55:23.057735 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.2:43319
--local_ip_for_outbound_sockets=127.8.154.2
--tserver_master_addrs=127.8.154.62:46845
--webserver_port=40719
--webserver_interface=127.8.154.2
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:39543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:55:23.057886 1868 ts_tablet_manager.cc:1428] T 662f0888fd3c4603b9d826d65f891eb3 P c17f5ac68cd74824828707b8ac593ffc: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
Aug 09 08:55:23 dist-test-slave-t3q3 krb5kdc[862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265723, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:55:23.063889 2033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:23.049800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:41287 (local address 127.8.154.62:46845)
0809 08:55:23.052534 (+ 2734us) server_negotiation.cc:206] Beginning negotiation
0809 08:55:23.052549 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 08:55:23.052580 (+ 31us) server_negotiation.cc:407] Connection header received
0809 08:55:23.052684 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:55:23.052689 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:55:23.052728 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:55:23.052895 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:55:23.053719 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:23.055119 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:23.056684 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:23.057295 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:23.057403 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:23.060211 (+ 2808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:55:23.060216 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:55:23.060216 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:55:23.060230 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:55:23.062090 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:55:23.062536 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:55:23.062540 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:55:23.062540 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:55:23.062636 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:55:23.062922 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:55:23.062923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:55:23.062923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:55:23.063167 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:55:23.063266 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 08:55:23.063577 (+ 311us) server_negotiation.cc:299] Negotiation successful
0809 08:55:23.063705 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2688,"thread_start_us":82,"threads_started":1}
I20260809 08:55:23.063900 2031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:23.049696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.1:41287)
0809 08:55:23.049875 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:23.049924 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 08:55:23.050393 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:55:23.052943 (+ 2550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:55:23.052947 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:55:23.053313 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:55:23.053565 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:55:23.053571 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:55:23.055266 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:55:23.055267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:55:23.056575 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:55:23.056578 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:55:23.057496 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:55:23.057497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:55:23.057628 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:23.058257 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:55:23.058267 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:55:23.060048 (+ 1781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:55:23.062196 (+ 2148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:55:23.062204 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:55:23.062206 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:55:23.062432 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:55:23.062711 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:55:23.062712 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:55:23.062712 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:55:23.062853 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:55:23.063246 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:55:23.063253 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:55:23.063395 (+ 142us) client_negotiation.cc:769] Sending connection context
0809 08:55:23.063501 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 08:55:23.063690 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":93,"threads_started":1}
I20260809 08:55:23.064782 2026 heartbeater.cc:347] Connected to a master server at 127.8.154.62:46845
I20260809 08:55:23.064908 2026 heartbeater.cc:464] Registering TS with master...
I20260809 08:55:23.065090 2026 heartbeater.cc:511] Master 127.8.154.62:46845 requested a full tablet report, sending...
I20260809 08:55:23.065825 1753 ts_manager.cc:194] Registered new tserver with Master: c17f5ac68cd74824828707b8ac593ffc (127.8.154.1:45617)
I20260809 08:55:23.066944 1753 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.1@KRBTEST.COM'} at 127.8.154.1:41287
I20260809 08:55:23.067787 2026 heartbeater.cc:503] Master 127.8.154.62:46845 was elected leader, sending a full tablet report...
I20260809 08:55:23.075527 2039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:23.066684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58870 (local address 127.8.154.1:45617)
0809 08:55:23.066847 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 08:55:23.066852 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:55:23.072367 (+ 5515us) server_negotiation.cc:407] Connection header received
0809 08:55:23.072554 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:55:23.072557 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:55:23.072573 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:55:23.072671 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:55:23.072824 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:23.073596 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:23.074584 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:23.075060 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:23.075160 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:23.075218 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 08:55:23.075397 (+ 179us) server_negotiation.cc:299] Negotiation successful
0809 08:55:23.075436 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":83,"threads_started":1}
I20260809 08:55:23.075515 2038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:23.066420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:45617 (local address 127.0.0.1:58870)
0809 08:55:23.072241 (+ 5821us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:23.072281 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 08:55:23.072424 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:55:23.072677 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:55:23.072680 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:55:23.072682 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:55:23.072746 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:55:23.072747 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:55:23.073689 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:55:23.073690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:55:23.074479 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:55:23.074482 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:55:23.075165 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:55:23.075166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:55:23.075252 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:23.075264 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 08:55:23.075379 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 08:55:23.075436 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5748,"thread_start_us":87,"threads_started":1}
I20260809 08:55:23.075956 1928 tablet_service.cc:1514] Processing DeleteTablet for tablet 662f0888fd3c4603b9d826d65f891eb3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:55:02 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:58870
I20260809 08:55:23.076766 2040 tablet_replica.cc:330] stopping tablet replica
I20260809 08:55:23.076882 2040 raft_consensus.cc:2229] T 662f0888fd3c4603b9d826d65f891eb3 P c17f5ac68cd74824828707b8ac593ffc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:55:23.076944 2040 raft_consensus.cc:2258] T 662f0888fd3c4603b9d826d65f891eb3 P c17f5ac68cd74824828707b8ac593ffc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:55:23.077282 2040 ts_tablet_manager.cc:1905] T 662f0888fd3c4603b9d826d65f891eb3 P c17f5ac68cd74824828707b8ac593ffc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:55:23.079850 2040 ts_tablet_manager.cc:1918] T 662f0888fd3c4603b9d826d65f891eb3 P c17f5ac68cd74824828707b8ac593ffc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:55:23.079931 2040 log.cc:1196] T 662f0888fd3c4603b9d826d65f891eb3 P c17f5ac68cd74824828707b8ac593ffc: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/wal/wals/662f0888fd3c4603b9d826d65f891eb3
I20260809 08:55:23.080206 2040 ts_tablet_manager.cc:1939] T 662f0888fd3c4603b9d826d65f891eb3 P c17f5ac68cd74824828707b8ac593ffc: Deleting consensus metadata
I20260809 08:55:23.080739 1730 catalog_manager.cc:4841] TS c17f5ac68cd74824828707b8ac593ffc (127.8.154.1:45617): tablet 662f0888fd3c4603b9d826d65f891eb3 (table test-table [id=ed644b5be9cf464fbf4a1c0014737c62]) successfully deleted
W20260809 08:55:23.149555 2037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:55:23.149784 2037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:55:23.149811 2037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:55:23.151019 2037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:55:23.151069 2037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:55:23.151098 2037 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.2
I20260809 08:55:23.152616 2037 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:39543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.2:43319
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.154.2
--webserver_port=40719
--tserver_master_addrs=127.8.154.62:46845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.2
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:55:23.152881 2037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:55:23.153174 2037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:55:23.158735 2052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:55:23.158735 2047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:55:23.158728 2050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:55:23.159142 2037 server_base.cc:1034] running on GCE node
I20260809 08:55:23.159399 2037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:55:23.159682 2037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:55:23.160898 2037 hybrid_clock.cc:648] HybridClock initialized: now 1786265723160874 us; error 56 us; skew 500 ppm
Aug 09 08:55:23 dist-test-slave-t3q3 krb5kdc[862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265723, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:55:23.163720 2037 init.cc:377] Logged in from keytab as kudu/127.8.154.2@KRBTEST.COM (short username kudu)
I20260809 08:55:23.164314 2037 webserver.cc:460] Webserver started at http://127.8.154.2:40719/ using document root <none> and password file <none>
I20260809 08:55:23.164588 2037 fs_manager.cc:359] Metadata directory not provided
I20260809 08:55:23.164683 2037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:55:23.166193 2037 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:55:23.166827 2064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:55:23.167021 2037 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:55:23.167155 2037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/wal
uuid: "d040e1f52a2743eaaccaa5e30c679477"
format_stamp: "Formatted at 2026-08-09 08:55:02 on dist-test-slave-t3q3"
server_key: "1084e86c51c5bf6ded8911231565fdad"
server_key_iv: "f3b21401e84290c0334f3d68eb6f17ea"
server_key_version: "encryptionkey@0"
I20260809 08:55:23.167517 2037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:55:23.186775 2037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:55:23.189216 2037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:55:23.189405 2037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:55:23.189688 2037 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:55:23.190030 2037 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:55:23.190088 2037 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:55:23.190176 2037 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:55:23.190209 2037 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:55:23.196280 2037 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.2:43319
I20260809 08:55:23.196395 2230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.2:43319 every 8 connection(s)
I20260809 08:55:23.196753 2037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
Aug 09 08:55:23 dist-test-slave-t3q3 krb5kdc[862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265723, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:55:23.205137 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 2037
I20260809 08:55:23.205508 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.3:39667
--local_ip_for_outbound_sockets=127.8.154.3
--tserver_master_addrs=127.8.154.62:46845
--webserver_port=33843
--webserver_interface=127.8.154.3
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:39543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:55:23.206765 2033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:23.197679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:48977 (local address 127.8.154.62:46845)
0809 08:55:23.197763 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 08:55:23.197765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:55:23.198016 (+ 251us) server_negotiation.cc:407] Connection header received
0809 08:55:23.198501 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:55:23.198502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:55:23.198515 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:55:23.198571 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:55:23.199558 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:23.200148 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:23.201218 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:23.201423 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:23.201488 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:23.203808 (+ 2320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:55:23.203809 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:55:23.203809 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:55:23.203816 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:55:23.205151 (+ 1335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:55:23.205711 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:55:23.205713 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:55:23.205713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:55:23.205777 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:55:23.206069 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:55:23.206070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:55:23.206070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:55:23.206197 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:55:23.206257 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 08:55:23.206621 (+ 364us) server_negotiation.cc:299] Negotiation successful
0809 08:55:23.206698 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 08:55:23.207084 2236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:23.197631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.2:48977)
0809 08:55:23.197872 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:23.197916 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 08:55:23.198412 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:55:23.198629 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:55:23.198633 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:55:23.199237 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:55:23.199445 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:55:23.199450 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:55:23.200244 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:55:23.200245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:55:23.201103 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:55:23.201106 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:55:23.201511 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:55:23.201512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:55:23.201628 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:23.202269 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:55:23.202278 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:55:23.203711 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:55:23.205238 (+ 1527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:55:23.205247 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:55:23.205249 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:55:23.205588 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:55:23.205853 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:55:23.205854 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:55:23.205854 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:55:23.205995 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:55:23.206272 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:55:23.206279 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:55:23.206471 (+ 192us) client_negotiation.cc:769] Sending connection context
0809 08:55:23.206593 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 08:55:23.206790 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":111,"threads_started":1}
I20260809 08:55:23.207633 2232 heartbeater.cc:347] Connected to a master server at 127.8.154.62:46845
I20260809 08:55:23.207751 2232 heartbeater.cc:464] Registering TS with master...
I20260809 08:55:23.207968 2232 heartbeater.cc:511] Master 127.8.154.62:46845 requested a full tablet report, sending...
I20260809 08:55:23.208474 1753 ts_manager.cc:194] Registered new tserver with Master: d040e1f52a2743eaaccaa5e30c679477 (127.8.154.2:43319)
I20260809 08:55:23.208966 1753 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.2@KRBTEST.COM'} at 127.8.154.2:48977
W20260809 08:55:23.290241 2237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:55:23.290442 2237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:55:23.290467 2237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:55:23.291523 2237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:55:23.291584 2237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:55:23.291643 2237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.3
I20260809 08:55:23.293124 2237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:39543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.3:39667
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.154.3
--webserver_port=33843
--tserver_master_addrs=127.8.154.62:46845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.3
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:55:23.293313 2237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:55:23.293481 2237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:55:23.298615 2246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:55:23.298663 2250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:55:23.298606 2247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:55:23.298815 2237 server_base.cc:1034] running on GCE node
I20260809 08:55:23.299019 2237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:55:23.299242 2237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:55:23.300416 2237 hybrid_clock.cc:648] HybridClock initialized: now 1786265723300407 us; error 27 us; skew 500 ppm
Aug 09 08:55:23 dist-test-slave-t3q3 krb5kdc[862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265723, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:55:23.302932 2237 init.cc:377] Logged in from keytab as kudu/127.8.154.3@KRBTEST.COM (short username kudu)
I20260809 08:55:23.303391 2237 webserver.cc:460] Webserver started at http://127.8.154.3:33843/ using document root <none> and password file <none>
I20260809 08:55:23.303575 2237 fs_manager.cc:359] Metadata directory not provided
I20260809 08:55:23.303638 2237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:55:23.305119 2237 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:55:23.305768 2259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:55:23.305986 2237 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:55:23.306098 2237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/wal
uuid: "52ae6da16ff7455aa1d0a0b491088b02"
format_stamp: "Formatted at 2026-08-09 08:55:02 on dist-test-slave-t3q3"
server_key: "f9f4dc8a50c7655cef9f0bba176e18b2"
server_key_iv: "5d6db3609d5452828c875d8047f31045"
server_key_version: "encryptionkey@0"
I20260809 08:55:23.306437 2237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:55:23.316351 2237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:55:23.318578 2237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:55:23.318742 2237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:55:23.318970 2237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:55:23.319275 2237 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:55:23.319329 2237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:55:23.319368 2237 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:55:23.319398 2237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:55:23.325605 2237 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.3:39667
I20260809 08:55:23.325665 2436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.3:39667 every 8 connection(s)
I20260809 08:55:23.325978 2237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
Aug 09 08:55:23 dist-test-slave-t3q3 krb5kdc[862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265723, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:55:23.332309 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 2237
I20260809 08:55:23.335374 2033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:23.326671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:60405 (local address 127.8.154.62:46845)
0809 08:55:23.326719 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 08:55:23.326720 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:55:23.326947 (+ 227us) server_negotiation.cc:407] Connection header received
0809 08:55:23.327418 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:55:23.327419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:55:23.327429 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:55:23.327482 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:55:23.328501 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:23.329038 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:23.330104 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:23.330294 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:23.330350 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:23.332516 (+ 2166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:55:23.332518 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:55:23.332518 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:55:23.332526 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:55:23.333984 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:55:23.334406 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:55:23.334407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:55:23.334408 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:55:23.334471 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:55:23.334731 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:55:23.334732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:55:23.334732 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:55:23.334846 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:55:23.334921 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 08:55:23.335205 (+ 284us) server_negotiation.cc:299] Negotiation successful
0809 08:55:23.335284 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 08:55:23.335702 2441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:23.326687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.3:60405)
0809 08:55:23.326843 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:23.326878 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 08:55:23.327331 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:55:23.327534 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:55:23.327537 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:55:23.328177 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:55:23.328421 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:55:23.328427 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:55:23.329123 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:55:23.329124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:55:23.330024 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:55:23.330026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:55:23.330373 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:55:23.330374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:55:23.330476 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:23.331018 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:55:23.331026 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:55:23.332430 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:55:23.334088 (+ 1658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:55:23.334103 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:55:23.334105 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:55:23.334326 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:55:23.334550 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:55:23.334551 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:55:23.334551 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:55:23.334663 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:55:23.334925 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:55:23.334932 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:55:23.335067 (+ 135us) client_negotiation.cc:769] Sending connection context
0809 08:55:23.335182 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 08:55:23.335337 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":94,"threads_started":1}
I20260809 08:55:23.336194 2439 heartbeater.cc:347] Connected to a master server at 127.8.154.62:46845
I20260809 08:55:23.336309 2439 heartbeater.cc:464] Registering TS with master...
I20260809 08:55:23.336516 2439 heartbeater.cc:511] Master 127.8.154.62:46845 requested a full tablet report, sending...
I20260809 08:55:23.336913 1753 ts_manager.cc:194] Registered new tserver with Master: 52ae6da16ff7455aa1d0a0b491088b02 (127.8.154.3:39667)
I20260809 08:55:23.337376 1753 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.3@KRBTEST.COM'} at 127.8.154.3:60405
I20260809 08:55:23.339389 2443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:23.332726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36142 (local address 127.8.154.62:46845)
0809 08:55:23.332851 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 08:55:23.332854 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:55:23.332980 (+ 126us) server_negotiation.cc:407] Connection header received
0809 08:55:23.333075 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:55:23.333076 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:55:23.333088 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:55:23.333168 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:55:23.333742 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:23.334211 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:23.334980 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:23.335176 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:23.335251 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:23.336762 (+ 1511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:55:23.336764 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:55:23.336764 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:55:23.336771 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:55:23.338423 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:55:23.338736 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:55:23.338736 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:55:23.338737 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:55:23.338788 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:55:23.338950 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:55:23.338950 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:55:23.338950 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:55:23.339033 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:55:23.339158 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 08:55:23.339249 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 08:55:23.339308 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":64,"threads_started":1}
I20260809 08:55:23.340049 8808 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:55:23.342571 2443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:23.340423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36144 (local address 127.8.154.62:46845)
0809 08:55:23.340480 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 08:55:23.340482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:55:23.340584 (+ 102us) server_negotiation.cc:407] Connection header received
0809 08:55:23.340642 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:55:23.340651 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:55:23.340678 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:55:23.340735 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:55:23.340848 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:23.341273 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:23.342001 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:23.342185 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:23.342238 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:23.342361 (+ 123us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:55:23.342399 (+ 38us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 08:55:23.342533 (+ 134us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36144: token expired
Metrics: {"server-negotiator.queue_time_us":38}
W20260809 08:55:23.342640 2443 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36144: token expired
W20260809 08:55:23.343488 1635 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.8.154.62:46845)
I20260809 08:55:23.349184 2443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:23.343743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36148 (local address 127.8.154.62:46845)
0809 08:55:23.343806 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 08:55:23.343808 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:55:23.343820 (+ 12us) server_negotiation.cc:407] Connection header received
0809 08:55:23.343840 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:55:23.343841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:55:23.343849 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:55:23.343898 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:55:23.344457 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:23.344897 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:23.345593 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:23.345790 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:23.345863 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:23.346901 (+ 1038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:55:23.346902 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:55:23.346902 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:55:23.346910 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:55:23.347996 (+ 1086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:55:23.348362 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:55:23.348363 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:55:23.348363 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:55:23.348424 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:55:23.348640 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:55:23.348641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:55:23.348641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:55:23.348779 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:55:23.348889 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 08:55:23.349006 (+ 117us) server_negotiation.cc:299] Negotiation successful
0809 08:55:23.349082 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 08:55:23.350286 1753 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36148:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:55:23.351759 1753 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:55:23.354024 1928 tablet_service.cc:1467] Processing CreateTablet for tablet c6592ba59a764d15ac993f2333858370 (DEFAULT_TABLE table=test-table [id=8486ba0f0f814921a04f0cd3421068a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:55:23.354238 1928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6592ba59a764d15ac993f2333858370. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:55:23.355826 1868 tablet_bootstrap.cc:492] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc: Bootstrap starting.
I20260809 08:55:23.356328 1868 tablet_bootstrap.cc:654] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc: Neither blocks nor log segments found. Creating new log.
I20260809 08:55:23.357051 1868 tablet_bootstrap.cc:492] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc: No bootstrap required, opened a new log
I20260809 08:55:23.357124 1868 ts_tablet_manager.cc:1397] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 08:55:23.357282 1868 raft_consensus.cc:348] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ae6da16ff7455aa1d0a0b491088b02" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 39667 } } peers { permanent_uuid: "d040e1f52a2743eaaccaa5e30c679477" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43319 } } peers { permanent_uuid: "c17f5ac68cd74824828707b8ac593ffc" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45617 } }
I20260809 08:55:23.357373 1868 raft_consensus.cc:374] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:55:23.357434 1868 raft_consensus.cc:729] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c17f5ac68cd74824828707b8ac593ffc, State: Initialized, Role: FOLLOWER
I20260809 08:55:23.357489 1868 consensus_queue.cc:260] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ae6da16ff7455aa1d0a0b491088b02" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 39667 } } peers { permanent_uuid: "d040e1f52a2743eaaccaa5e30c679477" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43319 } } peers { permanent_uuid: "c17f5ac68cd74824828707b8ac593ffc" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45617 } }
I20260809 08:55:23.357601 1868 ts_tablet_manager.cc:1428] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 08:55:23.358071 2038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:23.353720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:39667 (local address 127.0.0.1:48700)
0809 08:55:23.353766 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:23.353778 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:55:23.353813 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:55:23.354413 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:55:23.354414 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:55:23.354416 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:55:23.354456 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:55:23.354457 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:55:23.355700 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:55:23.355702 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:55:23.356629 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:55:23.356633 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:55:23.357818 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:55:23.357819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:55:23.357891 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:23.357894 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:55:23.357964 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 08:55:23.358007 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 08:55:23.358098 2450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:23.353997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48700 (local address 127.8.154.3:39667)
0809 08:55:23.354200 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 08:55:23.354205 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:55:23.354215 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:55:23.354252 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:55:23.354254 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:55:23.354268 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:55:23.354407 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:55:23.354524 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:23.355604 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:23.356723 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:23.357526 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:23.357599 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:23.357668 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 08:55:23.357982 (+ 314us) server_negotiation.cc:299] Negotiation successful
0809 08:55:23.358021 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":111,"threads_started":1}
I20260809 08:55:23.358709 2451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:23.354197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60254 (local address 127.8.154.2:43319)
0809 08:55:23.354449 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0809 08:55:23.354453 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:55:23.354463 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:55:23.354502 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:55:23.354504 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:55:23.354520 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:55:23.354610 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:55:23.354696 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:23.355719 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:23.357362 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:23.358398 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:23.358479 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:23.358537 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 08:55:23.358616 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 08:55:23.358646 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":97,"threads_started":1}
I20260809 08:55:23.358708 2449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:23.353747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:43319 (local address 127.0.0.1:60254)
0809 08:55:23.353896 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:23.353904 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 08:55:23.353946 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:55:23.354611 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:55:23.354611 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:55:23.354613 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:55:23.354647 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:55:23.354648 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:55:23.356136 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:55:23.356137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:55:23.357273 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:55:23.357276 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:55:23.358481 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:55:23.358481 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:55:23.358544 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:23.358547 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:55:23.358599 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 08:55:23.358635 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":100,"threads_started":1}
I20260809 08:55:23.358906 2334 tablet_service.cc:1467] Processing CreateTablet for tablet c6592ba59a764d15ac993f2333858370 (DEFAULT_TABLE table=test-table [id=8486ba0f0f814921a04f0cd3421068a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:55:23.359248 2334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6592ba59a764d15ac993f2333858370. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:55:23.359627 2134 tablet_service.cc:1467] Processing CreateTablet for tablet c6592ba59a764d15ac993f2333858370 (DEFAULT_TABLE table=test-table [id=8486ba0f0f814921a04f0cd3421068a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:55:23.359920 2134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c6592ba59a764d15ac993f2333858370. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:55:23.361989 2455 tablet_bootstrap.cc:492] T c6592ba59a764d15ac993f2333858370 P 52ae6da16ff7455aa1d0a0b491088b02: Bootstrap starting.
I20260809 08:55:23.362591 2456 tablet_bootstrap.cc:492] T c6592ba59a764d15ac993f2333858370 P d040e1f52a2743eaaccaa5e30c679477: Bootstrap starting.
I20260809 08:55:23.362789 2455 tablet_bootstrap.cc:654] T c6592ba59a764d15ac993f2333858370 P 52ae6da16ff7455aa1d0a0b491088b02: Neither blocks nor log segments found. Creating new log.
I20260809 08:55:23.363183 2456 tablet_bootstrap.cc:654] T c6592ba59a764d15ac993f2333858370 P d040e1f52a2743eaaccaa5e30c679477: Neither blocks nor log segments found. Creating new log.
I20260809 08:55:23.363356 2455 log.cc:824] T c6592ba59a764d15ac993f2333858370 P 52ae6da16ff7455aa1d0a0b491088b02: Log is configured to *not* fsync() on all Append() calls
I20260809 08:55:23.363778 2456 log.cc:824] T c6592ba59a764d15ac993f2333858370 P d040e1f52a2743eaaccaa5e30c679477: Log is configured to *not* fsync() on all Append() calls
I20260809 08:55:23.364255 2455 tablet_bootstrap.cc:492] T c6592ba59a764d15ac993f2333858370 P 52ae6da16ff7455aa1d0a0b491088b02: No bootstrap required, opened a new log
I20260809 08:55:23.364333 2455 ts_tablet_manager.cc:1397] T c6592ba59a764d15ac993f2333858370 P 52ae6da16ff7455aa1d0a0b491088b02: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:55:23.364626 2456 tablet_bootstrap.cc:492] T c6592ba59a764d15ac993f2333858370 P d040e1f52a2743eaaccaa5e30c679477: No bootstrap required, opened a new log
I20260809 08:55:23.364696 2456 ts_tablet_manager.cc:1397] T c6592ba59a764d15ac993f2333858370 P d040e1f52a2743eaaccaa5e30c679477: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:55:23.365871 2455 raft_consensus.cc:348] T c6592ba59a764d15ac993f2333858370 P 52ae6da16ff7455aa1d0a0b491088b02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ae6da16ff7455aa1d0a0b491088b02" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 39667 } } peers { permanent_uuid: "d040e1f52a2743eaaccaa5e30c679477" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43319 } } peers { permanent_uuid: "c17f5ac68cd74824828707b8ac593ffc" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45617 } }
I20260809 08:55:23.366034 2455 raft_consensus.cc:374] T c6592ba59a764d15ac993f2333858370 P 52ae6da16ff7455aa1d0a0b491088b02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:55:23.366082 2455 raft_consensus.cc:729] T c6592ba59a764d15ac993f2333858370 P 52ae6da16ff7455aa1d0a0b491088b02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52ae6da16ff7455aa1d0a0b491088b02, State: Initialized, Role: FOLLOWER
I20260809 08:55:23.366035 2456 raft_consensus.cc:348] T c6592ba59a764d15ac993f2333858370 P d040e1f52a2743eaaccaa5e30c679477 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ae6da16ff7455aa1d0a0b491088b02" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 39667 } } peers { permanent_uuid: "d040e1f52a2743eaaccaa5e30c679477" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43319 } } peers { permanent_uuid: "c17f5ac68cd74824828707b8ac593ffc" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45617 } }
I20260809 08:55:23.366201 2455 consensus_queue.cc:260] T c6592ba59a764d15ac993f2333858370 P 52ae6da16ff7455aa1d0a0b491088b02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ae6da16ff7455aa1d0a0b491088b02" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 39667 } } peers { permanent_uuid: "d040e1f52a2743eaaccaa5e30c679477" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43319 } } peers { permanent_uuid: "c17f5ac68cd74824828707b8ac593ffc" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45617 } }
I20260809 08:55:23.366237 2456 raft_consensus.cc:374] T c6592ba59a764d15ac993f2333858370 P d040e1f52a2743eaaccaa5e30c679477 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:55:23.366300 2456 raft_consensus.cc:729] T c6592ba59a764d15ac993f2333858370 P d040e1f52a2743eaaccaa5e30c679477 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d040e1f52a2743eaaccaa5e30c679477, State: Initialized, Role: FOLLOWER
I20260809 08:55:23.366438 2456 consensus_queue.cc:260] T c6592ba59a764d15ac993f2333858370 P d040e1f52a2743eaaccaa5e30c679477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ae6da16ff7455aa1d0a0b491088b02" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 39667 } } peers { permanent_uuid: "d040e1f52a2743eaaccaa5e30c679477" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43319 } } peers { permanent_uuid: "c17f5ac68cd74824828707b8ac593ffc" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45617 } }
I20260809 08:55:23.366477 2439 heartbeater.cc:503] Master 127.8.154.62:46845 was elected leader, sending a full tablet report...
I20260809 08:55:23.366485 2455 ts_tablet_manager.cc:1428] T c6592ba59a764d15ac993f2333858370 P 52ae6da16ff7455aa1d0a0b491088b02: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:55:23.366712 2456 ts_tablet_manager.cc:1428] T c6592ba59a764d15ac993f2333858370 P d040e1f52a2743eaaccaa5e30c679477: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:55:23.366741 2232 heartbeater.cc:503] Master 127.8.154.62:46845 was elected leader, sending a full tablet report...
W20260809 08:55:23.447561 2234 tablet.cc:2369] T c6592ba59a764d15ac993f2333858370 P d040e1f52a2743eaaccaa5e30c679477: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:55:23.495157 2464 raft_consensus.cc:482] T c6592ba59a764d15ac993f2333858370 P d040e1f52a2743eaaccaa5e30c679477 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:55:23.495289 2464 raft_consensus.cc:504] T c6592ba59a764d15ac993f2333858370 P d040e1f52a2743eaaccaa5e30c679477 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ae6da16ff7455aa1d0a0b491088b02" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 39667 } } peers { permanent_uuid: "d040e1f52a2743eaaccaa5e30c679477" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43319 } } peers { permanent_uuid: "c17f5ac68cd74824828707b8ac593ffc" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45617 } }
I20260809 08:55:23.495777 2464 leader_election.cc:288] T c6592ba59a764d15ac993f2333858370 P d040e1f52a2743eaaccaa5e30c679477 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52ae6da16ff7455aa1d0a0b491088b02 (127.8.154.3:39667), c17f5ac68cd74824828707b8ac593ffc (127.8.154.1:45617)
I20260809 08:55:23.498903 2450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:23.496137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:52559 (local address 127.8.154.3:39667)
0809 08:55:23.496184 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 08:55:23.496186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:55:23.496218 (+ 32us) server_negotiation.cc:407] Connection header received
0809 08:55:23.496336 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:55:23.496337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:55:23.496352 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:55:23.496421 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:55:23.496595 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:23.497092 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:23.498116 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:23.498516 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:23.498606 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:23.498618 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 08:55:23.498789 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 08:55:23.498831 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 08:55:23.498903 2236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:23.496025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:39667 (local address 127.8.154.2:52559)
0809 08:55:23.496155 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:23.496171 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 08:55:23.496238 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:55:23.496428 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:55:23.496429 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:55:23.496431 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:55:23.496512 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:55:23.496514 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:55:23.497184 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:55:23.497185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:55:23.498012 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:55:23.498015 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:55:23.498609 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:55:23.498610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:55:23.498697 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:23.498700 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:55:23.498779 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 08:55:23.498826 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"mutex_wait_us":23}
I20260809 08:55:23.498992 2039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:23.496137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:44253 (local address 127.8.154.1:45617)
0809 08:55:23.496192 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 08:55:23.496195 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:55:23.496434 (+ 239us) server_negotiation.cc:407] Connection header received
0809 08:55:23.496501 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:55:23.496502 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:55:23.496516 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:55:23.496570 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:55:23.496743 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:23.497226 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:23.498197 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:23.498604 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:23.498694 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:23.498704 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 08:55:23.498874 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 08:55:23.498918 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 08:55:23.499022 2466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:23.496041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:45617 (local address 127.8.154.2:44253)
0809 08:55:23.496341 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:23.496354 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:55:23.496442 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:55:23.496576 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:55:23.496577 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:55:23.496579 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:55:23.496661 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:55:23.496667 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:55:23.497318 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:55:23.497319 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:55:23.498095 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:55:23.498097 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:55:23.498700 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:55:23.498701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:55:23.498784 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:23.498788 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:55:23.498854 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 08:55:23.498899 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"mutex_wait_us":103,"thread_start_us":138,"threads_started":1}
I20260809 08:55:23.499402 2363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6592ba59a764d15ac993f2333858370" candidate_uuid: "d040e1f52a2743eaaccaa5e30c679477" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ae6da16ff7455aa1d0a0b491088b02" is_pre_election: true
I20260809 08:55:23.499554 2363 raft_consensus.cc:2454] T c6592ba59a764d15ac993f2333858370 P 52ae6da16ff7455aa1d0a0b491088b02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d040e1f52a2743eaaccaa5e30c679477 in term 0.
I20260809 08:55:23.499609 1962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6592ba59a764d15ac993f2333858370" candidate_uuid: "d040e1f52a2743eaaccaa5e30c679477" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c17f5ac68cd74824828707b8ac593ffc" is_pre_election: true
I20260809 08:55:23.499836 1962 raft_consensus.cc:2454] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d040e1f52a2743eaaccaa5e30c679477 in term 0.
I20260809 08:55:23.499851 2068 leader_election.cc:302] T c6592ba59a764d15ac993f2333858370 P d040e1f52a2743eaaccaa5e30c679477 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52ae6da16ff7455aa1d0a0b491088b02, d040e1f52a2743eaaccaa5e30c679477; no voters:
I20260809 08:55:23.499974 2464 raft_consensus.cc:2781] T c6592ba59a764d15ac993f2333858370 P d040e1f52a2743eaaccaa5e30c679477 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:55:23.500090 2464 raft_consensus.cc:482] T c6592ba59a764d15ac993f2333858370 P d040e1f52a2743eaaccaa5e30c679477 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:55:23.500133 2464 raft_consensus.cc:3037] T c6592ba59a764d15ac993f2333858370 P d040e1f52a2743eaaccaa5e30c679477 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:55:23.500895 2464 raft_consensus.cc:504] T c6592ba59a764d15ac993f2333858370 P d040e1f52a2743eaaccaa5e30c679477 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ae6da16ff7455aa1d0a0b491088b02" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 39667 } } peers { permanent_uuid: "d040e1f52a2743eaaccaa5e30c679477" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43319 } } peers { permanent_uuid: "c17f5ac68cd74824828707b8ac593ffc" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45617 } }
I20260809 08:55:23.501062 2464 leader_election.cc:288] T c6592ba59a764d15ac993f2333858370 P d040e1f52a2743eaaccaa5e30c679477 [CANDIDATE]: Term 1 election: Requested vote from peers 52ae6da16ff7455aa1d0a0b491088b02 (127.8.154.3:39667), c17f5ac68cd74824828707b8ac593ffc (127.8.154.1:45617)
I20260809 08:55:23.501217 1962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6592ba59a764d15ac993f2333858370" candidate_uuid: "d040e1f52a2743eaaccaa5e30c679477" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c17f5ac68cd74824828707b8ac593ffc"
I20260809 08:55:23.501236 2363 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c6592ba59a764d15ac993f2333858370" candidate_uuid: "d040e1f52a2743eaaccaa5e30c679477" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52ae6da16ff7455aa1d0a0b491088b02"
I20260809 08:55:23.501303 1962 raft_consensus.cc:3037] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:55:23.501328 2363 raft_consensus.cc:3037] T c6592ba59a764d15ac993f2333858370 P 52ae6da16ff7455aa1d0a0b491088b02 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:55:23.502089 2363 raft_consensus.cc:2454] T c6592ba59a764d15ac993f2333858370 P 52ae6da16ff7455aa1d0a0b491088b02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d040e1f52a2743eaaccaa5e30c679477 in term 1.
I20260809 08:55:23.502089 1962 raft_consensus.cc:2454] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d040e1f52a2743eaaccaa5e30c679477 in term 1.
I20260809 08:55:23.502315 2067 leader_election.cc:302] T c6592ba59a764d15ac993f2333858370 P d040e1f52a2743eaaccaa5e30c679477 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c17f5ac68cd74824828707b8ac593ffc, d040e1f52a2743eaaccaa5e30c679477; no voters:
I20260809 08:55:23.502415 2464 raft_consensus.cc:2781] T c6592ba59a764d15ac993f2333858370 P d040e1f52a2743eaaccaa5e30c679477 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:55:23.502584 2464 raft_consensus.cc:686] T c6592ba59a764d15ac993f2333858370 P d040e1f52a2743eaaccaa5e30c679477 [term 1 LEADER]: Becoming Leader. State: Replica: d040e1f52a2743eaaccaa5e30c679477, State: Running, Role: LEADER
I20260809 08:55:23.502709 2464 consensus_queue.cc:237] T c6592ba59a764d15ac993f2333858370 P d040e1f52a2743eaaccaa5e30c679477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ae6da16ff7455aa1d0a0b491088b02" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 39667 } } peers { permanent_uuid: "d040e1f52a2743eaaccaa5e30c679477" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43319 } } peers { permanent_uuid: "c17f5ac68cd74824828707b8ac593ffc" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45617 } }
I20260809 08:55:23.503574 1752 catalog_manager.cc:5495] T c6592ba59a764d15ac993f2333858370 P d040e1f52a2743eaaccaa5e30c679477 reported cstate change: term changed from 0 to 1, leader changed from <none> to d040e1f52a2743eaaccaa5e30c679477 (127.8.154.2). New cstate: current_term: 1 leader_uuid: "d040e1f52a2743eaaccaa5e30c679477" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ae6da16ff7455aa1d0a0b491088b02" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 39667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d040e1f52a2743eaaccaa5e30c679477" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43319 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c17f5ac68cd74824828707b8ac593ffc" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45617 } health_report { overall_health: UNKNOWN } } }
I20260809 08:55:23.544507 2451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:23.541825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60260 (local address 127.8.154.2:43319)
0809 08:55:23.541896 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 08:55:23.541898 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:55:23.541908 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:55:23.542001 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:55:23.542007 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:55:23.542030 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:55:23.542115 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:55:23.542257 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:23.542803 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:23.543595 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:23.543841 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:23.543916 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:23.544127 (+ 211us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:55:23.544242 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:55:23.544341 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 08:55:23.544366 (+ 25us) server_negotiation.cc:299] Negotiation successful
0809 08:55:23.544385 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 08:55:23.546433 1962 raft_consensus.cc:1261] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [term 1 FOLLOWER]: Refusing update from remote peer d040e1f52a2743eaaccaa5e30c679477: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:55:23.546432 2363 raft_consensus.cc:1261] T c6592ba59a764d15ac993f2333858370 P 52ae6da16ff7455aa1d0a0b491088b02 [term 1 FOLLOWER]: Refusing update from remote peer d040e1f52a2743eaaccaa5e30c679477: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:55:23.546828 2464 consensus_queue.cc:1035] T c6592ba59a764d15ac993f2333858370 P d040e1f52a2743eaaccaa5e30c679477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c17f5ac68cd74824828707b8ac593ffc" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:55:23.546979 2467 consensus_queue.cc:1035] T c6592ba59a764d15ac993f2333858370 P d040e1f52a2743eaaccaa5e30c679477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52ae6da16ff7455aa1d0a0b491088b02" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 39667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:55:23.549249 2471 mvcc.cc:205] Tried to move back new op lower bound from 7316544403643236352 to 7316544403467616256. Current Snapshot: MvccSnapshot[applied={T|T < 7316544403643236352}]
I20260809 08:55:23.550307 2472 mvcc.cc:205] Tried to move back new op lower bound from 7316544403643236352 to 7316544403467616256. Current Snapshot: MvccSnapshot[applied={T|T < 7316544403643236352}]
I20260809 08:55:23.551608 1753 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:55:23.551710 1753 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:55:23.552429 1753 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10: Sending DeleteTablet for 3 replicas of tablet c6592ba59a764d15ac993f2333858370
I20260809 08:55:23.552747 2134 tablet_service.cc:1514] Processing DeleteTablet for tablet c6592ba59a764d15ac993f2333858370 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:55:23 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:60254
I20260809 08:55:23.552819 2334 tablet_service.cc:1514] Processing DeleteTablet for tablet c6592ba59a764d15ac993f2333858370 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:55:23 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:48700
I20260809 08:55:23.553026 1928 tablet_service.cc:1514] Processing DeleteTablet for tablet c6592ba59a764d15ac993f2333858370 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:55:23 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:58870
I20260809 08:55:23.553054 2484 tablet_replica.cc:330] T c6592ba59a764d15ac993f2333858370 P 52ae6da16ff7455aa1d0a0b491088b02: stopping tablet replica
I20260809 08:55:23.553153 2040 tablet_replica.cc:330] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc: stopping tablet replica
I20260809 08:55:23.553188 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 1813
I20260809 08:55:23.553224 2040 raft_consensus.cc:2229] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:55:23.553275 2040 raft_consensus.cc:2258] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:55:23.553363 2484 raft_consensus.cc:2229] T c6592ba59a764d15ac993f2333858370 P 52ae6da16ff7455aa1d0a0b491088b02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:55:23.553510 2484 raft_consensus.cc:2258] T c6592ba59a764d15ac993f2333858370 P 52ae6da16ff7455aa1d0a0b491088b02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:55:23.561625 2484 ts_tablet_manager.cc:1905] T c6592ba59a764d15ac993f2333858370 P 52ae6da16ff7455aa1d0a0b491088b02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:55:23.564015 2484 ts_tablet_manager.cc:1918] T c6592ba59a764d15ac993f2333858370 P 52ae6da16ff7455aa1d0a0b491088b02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:55:23.564153 2484 log.cc:1196] T c6592ba59a764d15ac993f2333858370 P 52ae6da16ff7455aa1d0a0b491088b02: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/wal/wals/c6592ba59a764d15ac993f2333858370
I20260809 08:55:23.564544 2484 ts_tablet_manager.cc:1939] T c6592ba59a764d15ac993f2333858370 P 52ae6da16ff7455aa1d0a0b491088b02: Deleting consensus metadata
I20260809 08:55:23.565125 1732 catalog_manager.cc:4841] TS 52ae6da16ff7455aa1d0a0b491088b02 (127.8.154.3:39667): tablet c6592ba59a764d15ac993f2333858370 (table test-table [id=8486ba0f0f814921a04f0cd3421068a3]) successfully deleted
I20260809 08:55:23.568365 2482 tablet_replica.cc:330] T c6592ba59a764d15ac993f2333858370 P d040e1f52a2743eaaccaa5e30c679477: stopping tablet replica
I20260809 08:55:23.568504 2482 raft_consensus.cc:2229] T c6592ba59a764d15ac993f2333858370 P d040e1f52a2743eaaccaa5e30c679477 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:55:23.568621 2482 raft_consensus.cc:2258] T c6592ba59a764d15ac993f2333858370 P d040e1f52a2743eaaccaa5e30c679477 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:55:23.569137 2482 ts_tablet_manager.cc:1905] T c6592ba59a764d15ac993f2333858370 P d040e1f52a2743eaaccaa5e30c679477: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 08:55:23.570706 1730 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.154.1:45617 (error 108)
I20260809 08:55:23.570825 2482 ts_tablet_manager.cc:1918] T c6592ba59a764d15ac993f2333858370 P d040e1f52a2743eaaccaa5e30c679477: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:55:23.570915 2482 log.cc:1196] T c6592ba59a764d15ac993f2333858370 P d040e1f52a2743eaaccaa5e30c679477: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/wal/wals/c6592ba59a764d15ac993f2333858370
I20260809 08:55:23.571230 2482 ts_tablet_manager.cc:1939] T c6592ba59a764d15ac993f2333858370 P d040e1f52a2743eaaccaa5e30c679477: Deleting consensus metadata
I20260809 08:55:23.571441 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 2037
I20260809 08:55:23.576571 2449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:23.576391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:45617 (local address 127.0.0.1:58874)
0809 08:55:23.576470 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:23.576527 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.1:45617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 08:55:23.576746 1730 catalog_manager.cc:4584] TS c17f5ac68cd74824828707b8ac593ffc (127.8.154.1:45617): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c6592ba59a764d15ac993f2333858370: Network error: Client connection negotiation failed: client connection to 127.8.154.1:45617: connect: Connection refused (error 111)
I20260809 08:55:23.580016 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 2237
I20260809 08:55:23.580049 2449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:23.579925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:43319 (local address 127.0.0.1:60262)
0809 08:55:23.579986 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:23.580017 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.2:43319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 08:55:23.594350 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 1701
I20260809 08:55:43.614054 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:46845
--webserver_interface=127.8.154.62
--webserver_port=43855
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:39543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:46845
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:55:43.699283 2486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:55:43.699477 2486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:55:43.699505 2486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:55:43.700699 2486 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 08:55:43.700743 2486 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 08:55:43.700757 2486 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 08:55:43.700768 2486 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:55:43.700783 2486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:55:43.700832 2486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:55:43.700851 2486 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:55:43.700873 2486 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:55:43.702332 2486 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:39543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:46845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:46845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=43855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:55:43.702580 2486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:55:43.702802 2486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:55:43.707675 2499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:55:43.707726 2494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:55:43.707681 2496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:55:43.707870 2486 server_base.cc:1034] running on GCE node
I20260809 08:55:43.708261 2486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:55:43.708570 2486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:55:43.709769 2486 hybrid_clock.cc:648] HybridClock initialized: now 1786265743709743 us; error 37 us; skew 500 ppm
Aug 09 08:55:43 dist-test-slave-t3q3 krb5kdc[862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265743, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:55:43.716521 2486 init.cc:377] Logged in from keytab as kudu/127.8.154.62@KRBTEST.COM (short username kudu)
I20260809 08:55:43.717012 2486 webserver.cc:460] Webserver started at http://127.8.154.62:43855/ using document root <none> and password file <none>
I20260809 08:55:43.717234 2486 fs_manager.cc:359] Metadata directory not provided
I20260809 08:55:43.717278 2486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:55:43.718652 2486 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:55:43.719331 2507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:55:43.719512 2486 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:55:43.719619 2486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "75ab210e8bd44fafb02a3561a90adb10"
format_stamp: "Formatted at 2026-08-09 08:55:01 on dist-test-slave-t3q3"
server_key: "8b389bee1bba5c18ca13141e7ece957f"
server_key_iv: "3b3f285f6511c6ee661d89f023f36d80"
server_key_version: "encryptionkey@0"
I20260809 08:55:43.719985 2486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:55:43.735679 2486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:55:43.738073 2486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:55:43.738245 2486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:55:43.742506 2486 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:46845
I20260809 08:55:43.742547 2589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:46845 every 8 connection(s)
I20260809 08:55:43.742949 2486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:55:43.744468 2590 sys_catalog.cc:263] Verifying existing consensus state
I20260809 08:55:43.745307 2590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10: Bootstrap starting.
I20260809 08:55:43.750435 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 2486
I20260809 08:55:43.750809 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.1:45617
--local_ip_for_outbound_sockets=127.8.154.1
--tserver_master_addrs=127.8.154.62:46845
--webserver_port=43363
--webserver_interface=127.8.154.1
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:39543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:55:43.754516 2590 log.cc:824] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10: Log is configured to *not* fsync() on all Append() calls
I20260809 08:55:43.757064 2590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 08:55:43.757403 2590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10: Bootstrap complete.
I20260809 08:55:43.759675 2590 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75ab210e8bd44fafb02a3561a90adb10" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46845 } }
I20260809 08:55:43.759899 2590 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75ab210e8bd44fafb02a3561a90adb10, State: Initialized, Role: FOLLOWER
I20260809 08:55:43.760134 2590 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75ab210e8bd44fafb02a3561a90adb10" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46845 } }
I20260809 08:55:43.760208 2590 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:55:43.760237 2590 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:55:43.760304 2590 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [term 2 FOLLOWER]: Advancing to term 3
I20260809 08:55:43.761315 2590 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75ab210e8bd44fafb02a3561a90adb10" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46845 } }
I20260809 08:55:43.761408 2590 leader_election.cc:302] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75ab210e8bd44fafb02a3561a90adb10; no voters:
I20260809 08:55:43.761623 2590 leader_election.cc:288] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 08:55:43.761669 2596 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 08:55:43.761850 2596 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [term 3 LEADER]: Becoming Leader. State: Replica: 75ab210e8bd44fafb02a3561a90adb10, State: Running, Role: LEADER
I20260809 08:55:43.761919 2590 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:55:43.762041 2596 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75ab210e8bd44fafb02a3561a90adb10" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46845 } }
I20260809 08:55:43.762423 2599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "75ab210e8bd44fafb02a3561a90adb10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75ab210e8bd44fafb02a3561a90adb10" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46845 } } }
I20260809 08:55:43.762540 2599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [sys.catalog]: This master's current role is: LEADER
I20260809 08:55:43.762771 2608 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:55:43.762424 2596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75ab210e8bd44fafb02a3561a90adb10. Latest consensus state: current_term: 3 leader_uuid: "75ab210e8bd44fafb02a3561a90adb10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75ab210e8bd44fafb02a3561a90adb10" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46845 } } }
I20260809 08:55:43.762939 2596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10 [sys.catalog]: This master's current role is: LEADER
I20260809 08:55:43.763763 2608 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:55:43.763916 2608 catalog_manager.cc:1259] Loaded cluster ID: 500c1f97dbb24587b8b25ae17323d610
I20260809 08:55:43.764158 2608 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:55:43.765311 2608 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:55:43.765506 2608 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10: Loaded TSK: 0
I20260809 08:55:43.765545 2608 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 75ab210e8bd44fafb02a3561a90adb10: Loaded TSK: 1
F20260809 08:55:43.765779 2608 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f680e1739ad google::LogMessage::Fail() at ??:0
@ 0x7f680e17688a google::LogMessage::SendToLog() at ??:0
@ 0x7f680e1734de google::LogMessage::Flush() at ??:0
@ 0x7f680e174a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f68108e0104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f68108ddd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f68160cd1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f68160cd48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f68160cd594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f6816093c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f681609dad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f68100410d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f6810036954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f6810e9a6db start_thread at ??:0
@ 0x7f680cc7161f clone at ??:0
W20260809 08:55:43.852044 2593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:55:43.852384 2593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:55:43.852428 2593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:55:43.853684 2593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:55:43.853760 2593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:55:43.853809 2593 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.1
I20260809 08:55:43.855402 2593 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:39543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.1:45617
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.154.1
--webserver_port=43363
--tserver_master_addrs=127.8.154.62:46845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.1
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:55:43.855696 2593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:55:43.856169 2593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:55:43.861397 2649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:55:43.861603 2593 server_base.cc:1034] running on GCE node
W20260809 08:55:43.861415 2650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:55:43.861757 2653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:55:43.861987 2593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:55:43.862279 2593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:55:43.863448 2593 hybrid_clock.cc:648] HybridClock initialized: now 1786265743863428 us; error 37 us; skew 500 ppm
Aug 09 08:55:43 dist-test-slave-t3q3 krb5kdc[862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265743, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:55:43.866089 2593 init.cc:377] Logged in from keytab as kudu/127.8.154.1@KRBTEST.COM (short username kudu)
I20260809 08:55:43.866617 2593 webserver.cc:460] Webserver started at http://127.8.154.1:43363/ using document root <none> and password file <none>
I20260809 08:55:43.866885 2593 fs_manager.cc:359] Metadata directory not provided
I20260809 08:55:43.866932 2593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:55:43.868539 2593 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 08:55:43.869267 2666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:55:43.869473 2593 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:55:43.869583 2593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/wal
uuid: "c17f5ac68cd74824828707b8ac593ffc"
format_stamp: "Formatted at 2026-08-09 08:55:02 on dist-test-slave-t3q3"
server_key: "8a084a3118dae1161263b0b7dfb9ff6a"
server_key_iv: "fe473b1bc4075b391181955ffcea863b"
server_key_version: "encryptionkey@0"
I20260809 08:55:43.870054 2593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:55:43.891497 2593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:55:43.894572 2593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:55:43.894748 2593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:55:43.895033 2593 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:55:43.895613 2677 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 08:55:43.897244 2593 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 08:55:43.897308 2593 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 08:55:43.897351 2593 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 08:55:43.898345 2593 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 08:55:43.898396 2593 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 08:55:43.898494 2677 tablet_bootstrap.cc:492] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc: Bootstrap starting.
I20260809 08:55:43.906144 2593 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.1:45617
I20260809 08:55:43.906692 2593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
I20260809 08:55:43.907572 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 2593
I20260809 08:55:43.907936 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.2:43319
--local_ip_for_outbound_sockets=127.8.154.2
--tserver_master_addrs=127.8.154.62:46845
--webserver_port=40719
--webserver_interface=127.8.154.2
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:39543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:55:43.910099 2677 log.cc:824] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc: Log is configured to *not* fsync() on all Append() calls
I20260809 08:55:43.910480 2838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.1:45617 every 8 connection(s)
I20260809 08:55:43.912976 2677 tablet_bootstrap.cc:492] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 08:55:43.913417 2677 tablet_bootstrap.cc:492] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc: Bootstrap complete.
I20260809 08:55:43.914798 2677 ts_tablet_manager.cc:1397] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.008s
I20260809 08:55:43.916674 2677 raft_consensus.cc:348] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ae6da16ff7455aa1d0a0b491088b02" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 39667 } } peers { permanent_uuid: "d040e1f52a2743eaaccaa5e30c679477" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43319 } } peers { permanent_uuid: "c17f5ac68cd74824828707b8ac593ffc" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45617 } }
I20260809 08:55:43.917119 2677 raft_consensus.cc:729] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c17f5ac68cd74824828707b8ac593ffc, State: Initialized, Role: FOLLOWER
I20260809 08:55:43.917325 2677 consensus_queue.cc:260] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ae6da16ff7455aa1d0a0b491088b02" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 39667 } } peers { permanent_uuid: "d040e1f52a2743eaaccaa5e30c679477" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43319 } } peers { permanent_uuid: "c17f5ac68cd74824828707b8ac593ffc" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45617 } }
I20260809 08:55:43.917688 2677 ts_tablet_manager.cc:1428] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260809 08:55:43.983054 2842 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:55:43.907508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.1:32813)
0809 08:55:43.912354 (+ 4846us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:43.912403 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 08:55:43.913054 (+ 651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:55:43.982869 (+ 69815us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":4793,"thread_start_us":122,"threads_started":1}
W20260809 08:55:43.983331 2839 heartbeater.cc:644] Failed to heartbeat to 127.8.154.62:46845 (0 consecutive failures): Network error: Failed to ping master at 127.8.154.62:46845: Client connection negotiation failed: client connection to 127.8.154.62:46845: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 08:55:43.983721 2842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:43.983553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.1:33403)
0809 08:55:43.983625 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:43.983685 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 08:55:43.984148 2842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:43.984004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.1:55071)
0809 08:55:43.984084 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:43.984117 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 08:55:43.984304 2839 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 08:55:43.999819 2843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:55:44.000046 2843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:55:44.000159 2843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:55:44.001339 2843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:55:44.001382 2843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:55:44.001410 2843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.2
I20260809 08:55:44.002887 2843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:39543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.2:43319
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.154.2
--webserver_port=40719
--tserver_master_addrs=127.8.154.62:46845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.2
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:55:44.003147 2843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:55:44.003398 2843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:55:44.008687 2852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:55:44.008697 2855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:55:44.008689 2860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:55:44.009008 2843 server_base.cc:1034] running on GCE node
I20260809 08:55:44.009234 2843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:55:44.009495 2843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:55:44.010671 2843 hybrid_clock.cc:648] HybridClock initialized: now 1786265744010661 us; error 38 us; skew 500 ppm
Aug 09 08:55:44 dist-test-slave-t3q3 krb5kdc[862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265744, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:55:44.013396 2843 init.cc:377] Logged in from keytab as kudu/127.8.154.2@KRBTEST.COM (short username kudu)
I20260809 08:55:44.013861 2843 webserver.cc:460] Webserver started at http://127.8.154.2:40719/ using document root <none> and password file <none>
I20260809 08:55:44.014115 2843 fs_manager.cc:359] Metadata directory not provided
I20260809 08:55:44.014164 2843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:55:44.015537 2843 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:55:44.016265 2871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:55:44.016481 2843 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:55:44.016558 2843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/wal
uuid: "d040e1f52a2743eaaccaa5e30c679477"
format_stamp: "Formatted at 2026-08-09 08:55:02 on dist-test-slave-t3q3"
server_key: "1084e86c51c5bf6ded8911231565fdad"
server_key_iv: "f3b21401e84290c0334f3d68eb6f17ea"
server_key_version: "encryptionkey@0"
I20260809 08:55:44.016930 2843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:55:44.031046 2843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:55:44.033463 2843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:55:44.033619 2843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:55:44.033870 2843 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:55:44.034201 2843 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:55:44.034240 2843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:55:44.034261 2843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:55:44.034274 2843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:55:44.040402 2843 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.2:43319
I20260809 08:55:44.040448 3053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.2:43319 every 8 connection(s)
I20260809 08:55:44.040812 2843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
I20260809 08:55:44.041880 3058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:44.041497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.2:38315)
0809 08:55:44.041668 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:44.041740 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":85,"threads_started":1}
W20260809 08:55:44.042346 3054 heartbeater.cc:644] Failed to heartbeat to 127.8.154.62:46845 (0 consecutive failures): Network error: Failed to ping master at 127.8.154.62:46845: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
I20260809 08:55:44.042620 3058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:44.042532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.2:50379)
0809 08:55:44.042576 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:44.042603 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 08:55:44.042900 3058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:44.042823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.2:57515)
0809 08:55:44.042866 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:44.042887 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 08:55:44.043015 3054 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 08:55:44.048399 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 2843
I20260809 08:55:44.048724 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.3:39667
--local_ip_for_outbound_sockets=127.8.154.3
--tserver_master_addrs=127.8.154.62:46845
--webserver_port=33843
--webserver_interface=127.8.154.3
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:39543
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:55:44.132464 3059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:55:44.132668 3059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:55:44.132692 3059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:55:44.133843 3059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:55:44.133888 3059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:55:44.133940 3059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.3
I20260809 08:55:44.135375 3059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:39543
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.3:39667
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.154.3
--webserver_port=33843
--tserver_master_addrs=127.8.154.62:46845
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.3
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:55:44.135620 3059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:55:44.135859 3059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:55:44.140952 3069 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:55:44.141124 3059 server_base.cc:1034] running on GCE node
W20260809 08:55:44.140936 3072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:55:44.140934 3068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:55:44.141484 3059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:55:44.141755 3059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:55:44.142923 3059 hybrid_clock.cc:648] HybridClock initialized: now 1786265744142907 us; error 27 us; skew 500 ppm
Aug 09 08:55:44 dist-test-slave-t3q3 krb5kdc[862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265744, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:55:44.145444 3059 init.cc:377] Logged in from keytab as kudu/127.8.154.3@KRBTEST.COM (short username kudu)
I20260809 08:55:44.145887 3059 webserver.cc:460] Webserver started at http://127.8.154.3:33843/ using document root <none> and password file <none>
I20260809 08:55:44.146122 3059 fs_manager.cc:359] Metadata directory not provided
I20260809 08:55:44.146193 3059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:55:44.147562 3059 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:55:44.148250 3087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:55:44.148427 3059 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:55:44.148536 3059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/wal
uuid: "52ae6da16ff7455aa1d0a0b491088b02"
format_stamp: "Formatted at 2026-08-09 08:55:02 on dist-test-slave-t3q3"
server_key: "f9f4dc8a50c7655cef9f0bba176e18b2"
server_key_iv: "5d6db3609d5452828c875d8047f31045"
server_key_version: "encryptionkey@0"
I20260809 08:55:44.148883 3059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:55:44.177336 3059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:55:44.179395 3059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:55:44.179581 3059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:55:44.179860 3059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:55:44.180279 3059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:55:44.180339 3059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:55:44.180377 3059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:55:44.180426 3059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:55:44.186682 3059 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.3:39667
I20260809 08:55:44.186733 3243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.3:39667 every 8 connection(s)
I20260809 08:55:44.187126 3059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
I20260809 08:55:44.188253 3248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:44.187860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.3:50779)
0809 08:55:44.188019 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:44.188112 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":76,"threads_started":1}
W20260809 08:55:44.188689 3244 heartbeater.cc:644] Failed to heartbeat to 127.8.154.62:46845 (0 consecutive failures): Network error: Failed to ping master at 127.8.154.62:46845: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
I20260809 08:55:44.188958 3248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:44.188872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.3:58855)
0809 08:55:44.188915 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:44.188942 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 08:55:44.189237 3248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:44.189156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.3:59071)
0809 08:55:44.189197 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:44.189223 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 08:55:44.189365 3244 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 08:55:44.194872 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 3059
I20260809 08:55:44.984869 3251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:44.984637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.1:56141)
0809 08:55:44.984755 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:44.984791 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":64,"threads_started":1}
I20260809 08:55:45.043524 3252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:45.043322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.2:55265)
0809 08:55:45.043439 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:45.043473 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":58,"threads_started":1}
I20260809 08:55:45.189896 3253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:45.189669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.3:56241)
0809 08:55:45.189803 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:45.189845 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":81,"threads_started":1}
I20260809 08:55:45.459420 3254 raft_consensus.cc:482] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:55:45.459614 3254 raft_consensus.cc:504] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ae6da16ff7455aa1d0a0b491088b02" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 39667 } } peers { permanent_uuid: "d040e1f52a2743eaaccaa5e30c679477" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43319 } } peers { permanent_uuid: "c17f5ac68cd74824828707b8ac593ffc" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45617 } }
I20260809 08:55:45.459962 3254 leader_election.cc:288] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ae6da16ff7455aa1d0a0b491088b02 (127.8.154.3:39667), d040e1f52a2743eaaccaa5e30c679477 (127.8.154.2:43319)
Aug 09 08:55:45 dist-test-slave-t3q3 krb5kdc[862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265743, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for kudu/127.8.154.3@KRBTEST.COM
Aug 09 08:55:45 dist-test-slave-t3q3 krb5kdc[862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265743, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for kudu/127.8.154.2@KRBTEST.COM
I20260809 08:55:45.470935 3256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:45.460176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:39667 (local address 127.8.154.1:52291)
0809 08:55:45.460316 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:45.460325 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:55:45.460366 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:55:45.461198 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:55:45.461202 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:55:45.461572 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:55:45.461756 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:55:45.461763 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:55:45.462954 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:55:45.462955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:55:45.463700 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:55:45.463702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:55:45.464369 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:55:45.464370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:55:45.464495 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:45.465156 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:55:45.465160 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:55:45.466794 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:55:45.469032 (+ 2238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:55:45.469039 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:55:45.469041 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:55:45.469390 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:55:45.469677 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:55:45.469677 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:55:45.469678 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:55:45.469821 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:55:45.470302 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:55:45.470311 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:55:45.470477 (+ 166us) client_negotiation.cc:769] Sending connection context
0809 08:55:45.470603 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 08:55:45.470809 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":13,"thread_start_us":82,"threads_started":1}
I20260809 08:55:45.470949 3258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:45.460283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:37451 (local address 127.8.154.2:43319)
0809 08:55:45.460455 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 08:55:45.460464 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:55:45.460495 (+ 31us) server_negotiation.cc:407] Connection header received
0809 08:55:45.460937 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:55:45.460939 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:55:45.460981 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:55:45.461165 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:55:45.461890 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:45.462764 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:45.463807 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:45.464261 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:45.464392 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:45.467323 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:55:45.467326 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:55:45.467326 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:55:45.467336 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:55:45.469170 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:55:45.469580 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:55:45.469582 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:55:45.469583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:55:45.469691 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:55:45.469901 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:55:45.469902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:55:45.469902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:55:45.470220 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:55:45.470305 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 08:55:45.470690 (+ 385us) server_negotiation.cc:299] Negotiation successful
0809 08:55:45.470832 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":57,"threads_started":1}
I20260809 08:55:45.470949 3259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:45.460290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:52291 (local address 127.8.154.3:39667)
0809 08:55:45.460447 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 08:55:45.460457 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:55:45.460487 (+ 30us) server_negotiation.cc:407] Connection header received
0809 08:55:45.460948 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:55:45.460950 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:55:45.460987 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:55:45.461140 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:55:45.461993 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:45.462852 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:45.463794 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:55:45.464265 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:55:45.464395 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:45.466897 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:55:45.466901 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:55:45.466901 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:55:45.466928 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:55:45.468950 (+ 2022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:55:45.469472 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:55:45.469475 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:55:45.469475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:55:45.469595 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:55:45.469901 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:55:45.469902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:55:45.469902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:55:45.470215 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:55:45.470305 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 08:55:45.470700 (+ 395us) server_negotiation.cc:299] Negotiation successful
0809 08:55:45.470833 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":63,"threads_started":1}
I20260809 08:55:45.470954 3251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:45.460162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:43319 (local address 127.8.154.1:37451)
0809 08:55:45.460218 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:45.460231 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:55:45.460291 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:55:45.461198 (+ 907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:55:45.461202 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:55:45.461570 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:55:45.461756 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:55:45.461763 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:55:45.462870 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:55:45.462871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:55:45.463698 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:55:45.463700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:55:45.464365 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:55:45.464367 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:55:45.464498 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:55:45.465080 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:55:45.465090 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:55:45.467219 (+ 2129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:55:45.469247 (+ 2028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:55:45.469248 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:55:45.469248 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:55:45.469504 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:55:45.469764 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:55:45.469765 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:55:45.469765 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:55:45.469832 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:55:45.470302 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:55:45.470311 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:55:45.470477 (+ 166us) client_negotiation.cc:769] Sending connection context
0809 08:55:45.470603 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 08:55:45.470864 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":177}
W20260809 08:55:45.472596 2667 leader_election.cc:341] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d040e1f52a2743eaaccaa5e30c679477 (127.8.154.2:43319): Not found: Tablet not found: c6592ba59a764d15ac993f2333858370
W20260809 08:55:45.472721 2674 leader_election.cc:341] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 52ae6da16ff7455aa1d0a0b491088b02 (127.8.154.3:39667): Not found: Tablet not found: c6592ba59a764d15ac993f2333858370
I20260809 08:55:45.472759 2674 leader_election.cc:302] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c17f5ac68cd74824828707b8ac593ffc; no voters: 52ae6da16ff7455aa1d0a0b491088b02, d040e1f52a2743eaaccaa5e30c679477
I20260809 08:55:45.472867 3254 raft_consensus.cc:2726] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 08:55:45.985550 3261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:45.985345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.1:43467)
0809 08:55:45.985462 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:45.985501 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
I20260809 08:55:46.044301 3262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:46.044035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.2:34855)
0809 08:55:46.044219 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:46.044255 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":121,"threads_started":1}
I20260809 08:55:46.190743 3264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:46.190529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.3:34969)
0809 08:55:46.190662 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:46.190700 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":84,"threads_started":1}
I20260809 08:55:46.986372 3265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:46.986134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.1:51395)
0809 08:55:46.986278 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:46.986321 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":70,"threads_started":1}
I20260809 08:55:47.044976 3267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:47.044774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.2:50141)
0809 08:55:47.044896 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:47.044934 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":63,"threads_started":1}
I20260809 08:55:47.191443 3269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:47.191206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.3:49487)
0809 08:55:47.191365 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:47.191404 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":81,"threads_started":1}
I20260809 08:55:47.520289 3270 raft_consensus.cc:482] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:55:47.520478 3270 raft_consensus.cc:504] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ae6da16ff7455aa1d0a0b491088b02" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 39667 } } peers { permanent_uuid: "d040e1f52a2743eaaccaa5e30c679477" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43319 } } peers { permanent_uuid: "c17f5ac68cd74824828707b8ac593ffc" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45617 } }
I20260809 08:55:47.520699 3270 leader_election.cc:288] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ae6da16ff7455aa1d0a0b491088b02 (127.8.154.3:39667), d040e1f52a2743eaaccaa5e30c679477 (127.8.154.2:43319)
W20260809 08:55:47.521155 2667 leader_election.cc:341] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d040e1f52a2743eaaccaa5e30c679477 (127.8.154.2:43319): Not found: Tablet not found: c6592ba59a764d15ac993f2333858370
W20260809 08:55:47.521227 2674 leader_election.cc:341] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 52ae6da16ff7455aa1d0a0b491088b02 (127.8.154.3:39667): Not found: Tablet not found: c6592ba59a764d15ac993f2333858370
I20260809 08:55:47.521271 2674 leader_election.cc:302] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c17f5ac68cd74824828707b8ac593ffc; no voters: 52ae6da16ff7455aa1d0a0b491088b02, d040e1f52a2743eaaccaa5e30c679477
I20260809 08:55:47.521354 3270 raft_consensus.cc:2726] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 08:55:47.987160 3272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:47.986904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.1:40385)
0809 08:55:47.987060 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:47.987107 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":72,"threads_started":1}
I20260809 08:55:48.045691 3274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:48.045463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.2:50219)
0809 08:55:48.045611 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:48.045649 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":78,"threads_started":1}
I20260809 08:55:48.192220 3277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:48.191939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.3:40657)
0809 08:55:48.192137 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:48.192177 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":63,"threads_started":1}
I20260809 08:55:48.987849 3278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:48.987624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.1:38853)
0809 08:55:48.987762 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:48.987801 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":60,"threads_started":1}
I20260809 08:55:49.046501 3279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:49.046295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.2:51781)
0809 08:55:49.046420 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:49.046459 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":68,"threads_started":1}
I20260809 08:55:49.192890 3281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:49.192666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.3:56553)
0809 08:55:49.192810 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:49.192849 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":86,"threads_started":1}
I20260809 08:55:49.914561 3282 raft_consensus.cc:482] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:55:49.914675 3282 raft_consensus.cc:504] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ae6da16ff7455aa1d0a0b491088b02" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 39667 } } peers { permanent_uuid: "d040e1f52a2743eaaccaa5e30c679477" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43319 } } peers { permanent_uuid: "c17f5ac68cd74824828707b8ac593ffc" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45617 } }
I20260809 08:55:49.915042 3282 leader_election.cc:288] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ae6da16ff7455aa1d0a0b491088b02 (127.8.154.3:39667), d040e1f52a2743eaaccaa5e30c679477 (127.8.154.2:43319)
W20260809 08:55:49.915474 2674 leader_election.cc:341] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 52ae6da16ff7455aa1d0a0b491088b02 (127.8.154.3:39667): Not found: Tablet not found: c6592ba59a764d15ac993f2333858370
W20260809 08:55:49.915581 2667 leader_election.cc:341] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d040e1f52a2743eaaccaa5e30c679477 (127.8.154.2:43319): Not found: Tablet not found: c6592ba59a764d15ac993f2333858370
I20260809 08:55:49.915663 2667 leader_election.cc:302] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c17f5ac68cd74824828707b8ac593ffc; no voters: 52ae6da16ff7455aa1d0a0b491088b02, d040e1f52a2743eaaccaa5e30c679477
I20260809 08:55:49.915751 3282 raft_consensus.cc:2726] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 08:55:49.988651 3284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:49.988421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.1:38509)
0809 08:55:49.988562 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:49.988604 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":69,"threads_started":1}
I20260809 08:55:50.047179 3285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:50.046956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.2:60961)
0809 08:55:50.047089 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:50.047135 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
I20260809 08:55:50.193567 3287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:50.193342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.3:36161)
0809 08:55:50.193484 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:50.193521 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":66,"threads_started":1}
I20260809 08:55:50.989408 3288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:50.989152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.1:51243)
0809 08:55:50.989293 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:50.989336 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":63,"threads_started":1}
I20260809 08:55:51.047920 3290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:51.047686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.2:39813)
0809 08:55:51.047840 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:51.047876 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":84,"threads_started":1}
I20260809 08:55:51.194285 3292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:51.194067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.3:52745)
0809 08:55:51.194203 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:51.194240 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":65,"threads_started":1}
I20260809 08:55:51.990182 3293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:51.989932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.1:35699)
0809 08:55:51.990081 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:51.990132 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":84,"threads_started":1}
I20260809 08:55:52.048786 3294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:52.048544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.2:43657)
0809 08:55:52.048693 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:52.048739 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":77,"threads_started":1}
I20260809 08:55:52.195118 3295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:52.194836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.3:34567)
0809 08:55:52.195010 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:52.195071 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":89,"threads_started":1}
I20260809 08:55:52.990952 3296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:52.990699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.1:45553)
0809 08:55:52.990861 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:52.990902 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":80,"threads_started":1}
I20260809 08:55:53.049474 3298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:53.049262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.2:57675)
0809 08:55:53.049393 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:53.049429 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":62,"threads_started":1}
I20260809 08:55:53.195824 3300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:53.195619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.3:56441)
0809 08:55:53.195741 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:53.195777 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":63,"threads_started":1}
I20260809 08:55:53.991699 3302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:53.991419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.1:34171)
0809 08:55:53.991613 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:53.991650 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":89,"threads_started":1}
I20260809 08:55:54.050184 3304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:54.049942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.2:35307)
0809 08:55:54.050091 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:54.050136 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":69,"threads_started":1}
I20260809 08:55:54.196553 3305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:54.196310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.3:45665)
0809 08:55:54.196457 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:54.196507 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":81,"threads_started":1}
I20260809 08:55:54.721889 3306 raft_consensus.cc:482] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:55:54.722018 3306 raft_consensus.cc:504] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ae6da16ff7455aa1d0a0b491088b02" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 39667 } } peers { permanent_uuid: "d040e1f52a2743eaaccaa5e30c679477" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43319 } } peers { permanent_uuid: "c17f5ac68cd74824828707b8ac593ffc" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45617 } }
I20260809 08:55:54.722226 3306 leader_election.cc:288] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ae6da16ff7455aa1d0a0b491088b02 (127.8.154.3:39667), d040e1f52a2743eaaccaa5e30c679477 (127.8.154.2:43319)
W20260809 08:55:54.722653 2674 leader_election.cc:341] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 52ae6da16ff7455aa1d0a0b491088b02 (127.8.154.3:39667): Not found: Tablet not found: c6592ba59a764d15ac993f2333858370
W20260809 08:55:54.722764 2667 leader_election.cc:341] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d040e1f52a2743eaaccaa5e30c679477 (127.8.154.2:43319): Not found: Tablet not found: c6592ba59a764d15ac993f2333858370
I20260809 08:55:54.722828 2667 leader_election.cc:302] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c17f5ac68cd74824828707b8ac593ffc; no voters: 52ae6da16ff7455aa1d0a0b491088b02, d040e1f52a2743eaaccaa5e30c679477
I20260809 08:55:54.722894 3306 raft_consensus.cc:2726] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 08:55:54.992463 3307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:54.992227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.1:33827)
0809 08:55:54.992379 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:54.992415 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":85,"threads_started":1}
I20260809 08:55:55.050896 3310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:55.050670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.2:40531)
0809 08:55:55.050806 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:55.050854 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":64,"threads_started":1}
I20260809 08:55:55.197294 3311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:55.197043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.3:48273)
0809 08:55:55.197204 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:55.197248 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":66,"threads_started":1}
I20260809 08:55:55.993211 3312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:55.992975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.1:54767)
0809 08:55:55.993122 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:55.993157 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":67,"threads_started":1}
I20260809 08:55:56.051620 3313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:56.051346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.2:35059)
0809 08:55:56.051541 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:56.051577 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":97,"threads_started":1}
I20260809 08:55:56.198133 3315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:56.197855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.3:41339)
0809 08:55:56.198042 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:56.198083 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":90,"threads_started":1}
I20260809 08:55:56.994033 3317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:56.993755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.1:59421)
0809 08:55:56.993930 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:56.993978 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":89,"threads_started":1}
I20260809 08:55:57.052379 3318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:57.052168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.2:38849)
0809 08:55:57.052302 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:57.052338 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":61,"threads_started":1}
I20260809 08:55:57.198932 3319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:57.198681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.3:50859)
0809 08:55:57.198847 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:57.198887 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":66,"threads_started":1}
I20260809 08:55:57.994894 3320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:57.994558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.1:47785)
0809 08:55:57.994778 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:57.994844 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":130,"threads_started":1}
I20260809 08:55:58.053077 3321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:58.052811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.2:37489)
0809 08:55:58.052978 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:58.053028 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":85,"threads_started":1}
I20260809 08:55:58.143514 3323 raft_consensus.cc:482] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:55:58.143625 3323 raft_consensus.cc:504] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52ae6da16ff7455aa1d0a0b491088b02" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 39667 } } peers { permanent_uuid: "d040e1f52a2743eaaccaa5e30c679477" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 43319 } } peers { permanent_uuid: "c17f5ac68cd74824828707b8ac593ffc" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 45617 } }
I20260809 08:55:58.143834 3323 leader_election.cc:288] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 52ae6da16ff7455aa1d0a0b491088b02 (127.8.154.3:39667), d040e1f52a2743eaaccaa5e30c679477 (127.8.154.2:43319)
W20260809 08:55:58.144281 2667 leader_election.cc:341] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d040e1f52a2743eaaccaa5e30c679477 (127.8.154.2:43319): Not found: Tablet not found: c6592ba59a764d15ac993f2333858370
W20260809 08:55:58.144363 2674 leader_election.cc:341] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 52ae6da16ff7455aa1d0a0b491088b02 (127.8.154.3:39667): Not found: Tablet not found: c6592ba59a764d15ac993f2333858370
I20260809 08:55:58.144452 2674 leader_election.cc:302] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c17f5ac68cd74824828707b8ac593ffc; no voters: 52ae6da16ff7455aa1d0a0b491088b02, d040e1f52a2743eaaccaa5e30c679477
I20260809 08:55:58.144541 3323 raft_consensus.cc:2726] T c6592ba59a764d15ac993f2333858370 P c17f5ac68cd74824828707b8ac593ffc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 08:55:58.199621 3324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:58.199385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.3:47945)
0809 08:55:58.199528 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:58.199566 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":83,"threads_started":1}
I20260809 08:55:58.995672 3325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:58.995464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.1:55109)
0809 08:55:58.995595 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:58.995626 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":76,"threads_started":1}
I20260809 08:55:59.053802 3327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:59.053580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.2:40997)
0809 08:55:59.053714 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:59.053755 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":59,"threads_started":1}
I20260809 08:55:59.200414 3329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:55:59.200196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46845 (local address 127.8.154.3:58883)
0809 08:55:59.200337 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 08:55:59.200372 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.62:46845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":82,"threads_started":1}
I20260809 08:55:59.201323 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 2593
I20260809 08:55:59.216475 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 2843
I20260809 08:55:59.225363 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 3059
2026-08-09T08:55:59Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58807 ms)
[----------] 30 tests from SecurityITest (239280 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:55:59 dist-test-slave-t3q3 krb5kdc[3332](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:55:59 dist-test-slave-t3q3 krb5kdc[3332](info): set up 2 sockets
Aug 09 08:55:59 dist-test-slave-t3q3 krb5kdc[3332](info): commencing operation
krb5kdc: starting...
W20260809 08:56:00.468411 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.218s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:56:00 dist-test-slave-t3q3 krb5kdc[3332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:56:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:56:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
I20260809 08:56:00.587039 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:35361
--webserver_interface=127.8.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:39077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:35361
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:56:00.668437 3348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:00.668689 3348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:00.668726 3348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:00.669847 3348 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:56:00.669917 3348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:00.669943 3348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:00.669965 3348 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:56:00.669993 3348 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:56:00.671469 3348 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:39077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:35361
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:35361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:00.671751 3348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:00.671991 3348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:00.677410 3358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:00.677567 3348 server_base.cc:1034] running on GCE node
W20260809 08:56:00.677547 3356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:00.677416 3355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:00.678052 3348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:00.678376 3348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:00.679553 3348 hybrid_clock.cc:648] HybridClock initialized: now 1786265760679539 us; error 32 us; skew 500 ppm
Aug 09 08:56:00 dist-test-slave-t3q3 krb5kdc[3332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265760, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:56:00.682197 3348 init.cc:377] Logged in from keytab as kudu/127.8.154.62@KRBTEST.COM (short username kudu)
I20260809 08:56:00.682698 3348 webserver.cc:460] Webserver started at http://127.8.154.62:36231/ using document root <none> and password file <none>
I20260809 08:56:00.682924 3348 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:00.682988 3348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:00.683140 3348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:00.684309 3348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "257ff16d7c524971a65bf1860994860f"
format_stamp: "Formatted at 2026-08-09 08:56:00 on dist-test-slave-t3q3"
server_key: "e6c2b0b1fc7ca75cb7f85d04e93e42c1"
server_key_iv: "f95839331e535024ac6963e3a9fe1a04"
server_key_version: "encryptionkey@0"
I20260809 08:56:00.684685 3348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "257ff16d7c524971a65bf1860994860f"
format_stamp: "Formatted at 2026-08-09 08:56:00 on dist-test-slave-t3q3"
server_key: "e6c2b0b1fc7ca75cb7f85d04e93e42c1"
server_key_iv: "f95839331e535024ac6963e3a9fe1a04"
server_key_version: "encryptionkey@0"
I20260809 08:56:00.686195 3348 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:56:00.686942 3371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:00.687161 3348 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:56:00.687265 3348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "257ff16d7c524971a65bf1860994860f"
format_stamp: "Formatted at 2026-08-09 08:56:00 on dist-test-slave-t3q3"
server_key: "e6c2b0b1fc7ca75cb7f85d04e93e42c1"
server_key_iv: "f95839331e535024ac6963e3a9fe1a04"
server_key_version: "encryptionkey@0"
I20260809 08:56:00.687343 3348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:00.705355 3348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:00.707161 3348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:00.707325 3348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:00.711513 3348 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:35361
I20260809 08:56:00.711555 3450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:35361 every 8 connection(s)
I20260809 08:56:00.711906 3348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:56:00.712783 3451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:00.715149 3451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 257ff16d7c524971a65bf1860994860f: Bootstrap starting.
I20260809 08:56:00.715742 3451 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 257ff16d7c524971a65bf1860994860f: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:00.716033 3451 log.cc:824] T 00000000000000000000000000000000 P 257ff16d7c524971a65bf1860994860f: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:00.716755 3451 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 257ff16d7c524971a65bf1860994860f: No bootstrap required, opened a new log
I20260809 08:56:00.718313 3451 raft_consensus.cc:348] T 00000000000000000000000000000000 P 257ff16d7c524971a65bf1860994860f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "257ff16d7c524971a65bf1860994860f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 35361 } }
I20260809 08:56:00.718473 3451 raft_consensus.cc:374] T 00000000000000000000000000000000 P 257ff16d7c524971a65bf1860994860f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:00.718525 3451 raft_consensus.cc:729] T 00000000000000000000000000000000 P 257ff16d7c524971a65bf1860994860f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 257ff16d7c524971a65bf1860994860f, State: Initialized, Role: FOLLOWER
I20260809 08:56:00.718679 3451 consensus_queue.cc:260] T 00000000000000000000000000000000 P 257ff16d7c524971a65bf1860994860f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "257ff16d7c524971a65bf1860994860f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 35361 } }
I20260809 08:56:00.718761 3451 raft_consensus.cc:388] T 00000000000000000000000000000000 P 257ff16d7c524971a65bf1860994860f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:56:00.718806 3451 raft_consensus.cc:482] T 00000000000000000000000000000000 P 257ff16d7c524971a65bf1860994860f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:56:00.718858 3451 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 257ff16d7c524971a65bf1860994860f [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:00.719403 3451 raft_consensus.cc:504] T 00000000000000000000000000000000 P 257ff16d7c524971a65bf1860994860f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "257ff16d7c524971a65bf1860994860f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 35361 } }
I20260809 08:56:00.719530 3451 leader_election.cc:302] T 00000000000000000000000000000000 P 257ff16d7c524971a65bf1860994860f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 257ff16d7c524971a65bf1860994860f; no voters:
I20260809 08:56:00.719710 3451 leader_election.cc:288] T 00000000000000000000000000000000 P 257ff16d7c524971a65bf1860994860f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:56:00.719749 3457 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 257ff16d7c524971a65bf1860994860f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:56:00.719920 3457 raft_consensus.cc:686] T 00000000000000000000000000000000 P 257ff16d7c524971a65bf1860994860f [term 1 LEADER]: Becoming Leader. State: Replica: 257ff16d7c524971a65bf1860994860f, State: Running, Role: LEADER
I20260809 08:56:00.719980 3451 sys_catalog.cc:564] T 00000000000000000000000000000000 P 257ff16d7c524971a65bf1860994860f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:56:00.720118 3457 consensus_queue.cc:237] T 00000000000000000000000000000000 P 257ff16d7c524971a65bf1860994860f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "257ff16d7c524971a65bf1860994860f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 35361 } }
I20260809 08:56:00.720542 3459 sys_catalog.cc:455] T 00000000000000000000000000000000 P 257ff16d7c524971a65bf1860994860f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 257ff16d7c524971a65bf1860994860f. Latest consensus state: current_term: 1 leader_uuid: "257ff16d7c524971a65bf1860994860f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "257ff16d7c524971a65bf1860994860f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 35361 } } }
I20260809 08:56:00.720638 3459 sys_catalog.cc:458] T 00000000000000000000000000000000 P 257ff16d7c524971a65bf1860994860f [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:00.720685 3458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 257ff16d7c524971a65bf1860994860f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "257ff16d7c524971a65bf1860994860f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "257ff16d7c524971a65bf1860994860f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 35361 } } }
I20260809 08:56:00.720752 3458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 257ff16d7c524971a65bf1860994860f [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:00.720886 3470 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:56:00.721565 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 3348
I20260809 08:56:00.721603 3470 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:56:00.721689 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:56:00.721918 8808 external_mini_cluster.cc:1428] Setting key cce89a9bd6568d769dd2772ec31468eb
I20260809 08:56:00.723378 3470 catalog_manager.cc:1347] Generated new cluster ID: 1d6552e07bba40ee98f83ada65d8691c
I20260809 08:56:00.723434 3470 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 08:56:00 dist-test-slave-t3q3 krb5kdc[3332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265760, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:56:00.734411 3489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:00.722404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42370 (local address 127.8.154.62:35361)
0809 08:56:00.722624 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:00.722632 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:00.722819 (+ 187us) server_negotiation.cc:407] Connection header received
0809 08:56:00.723065 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:00.723073 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:00.723135 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:00.724091 (+ 956us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:00.724187 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:00.725628 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:00.726691 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:00.727307 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:00.727707 (+ 400us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:00.730133 (+ 2426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:00.730137 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:00.730137 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:00.730150 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:00.732155 (+ 2005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:00.732527 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:00.732530 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:00.732530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:00.732638 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:00.732826 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:00.732826 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:00.732827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:00.733138 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:00.733922 (+ 784us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:00.733999 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 08:56:00.734154 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":177,"thread_start_us":70,"threads_started":1}
I20260809 08:56:00.740778 3470 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:56:00.741274 3470 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:56:00.743204 3470 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 257ff16d7c524971a65bf1860994860f: Generated new TSK 0
I20260809 08:56:00.743382 3470 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab.
Entry for principal kudu/127.8.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab.
I20260809 08:56:00.772557 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.1:0
--local_ip_for_outbound_sockets=127.8.154.1
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:35361
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:39077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:56:00.856412 3493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:00.856649 3493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:00.856690 3493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:00.857805 3493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:00.857867 3493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:00.857913 3493 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.1
I20260809 08:56:00.859426 3493 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:39077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:35361
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.1
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:00.859688 3493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:00.859911 3493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:00.864874 3508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:00.864876 3504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:00.864913 3506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:00.865216 3493 server_base.cc:1034] running on GCE node
I20260809 08:56:00.865495 3493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:00.865756 3493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:00.866930 3493 hybrid_clock.cc:648] HybridClock initialized: now 1786265760866916 us; error 37 us; skew 500 ppm
Aug 09 08:56:00 dist-test-slave-t3q3 krb5kdc[3332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265760, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:56:00.869841 3493 init.cc:377] Logged in from keytab as kudu/127.8.154.1@KRBTEST.COM (short username kudu)
I20260809 08:56:00.870322 3493 webserver.cc:460] Webserver started at http://127.8.154.1:35359/ using document root <none> and password file <none>
I20260809 08:56:00.870568 3493 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:00.870639 3493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:00.870798 3493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:00.871919 3493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-0/data/instance:
uuid: "32f61fade8324186a304c52f09b3455a"
format_stamp: "Formatted at 2026-08-09 08:56:00 on dist-test-slave-t3q3"
server_key: "04c2c51993d54f8dcc5144e20cb124e3"
server_key_iv: "bf328d6458cb871d8cd78eddc49ce0f0"
server_key_version: "encryptionkey@0"
I20260809 08:56:00.872406 3493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance:
uuid: "32f61fade8324186a304c52f09b3455a"
format_stamp: "Formatted at 2026-08-09 08:56:00 on dist-test-slave-t3q3"
server_key: "04c2c51993d54f8dcc5144e20cb124e3"
server_key_iv: "bf328d6458cb871d8cd78eddc49ce0f0"
server_key_version: "encryptionkey@0"
I20260809 08:56:00.874025 3493 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:56:00.874759 3518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:00.874970 3493 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:56:00.875104 3493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-0/wal
uuid: "32f61fade8324186a304c52f09b3455a"
format_stamp: "Formatted at 2026-08-09 08:56:00 on dist-test-slave-t3q3"
server_key: "04c2c51993d54f8dcc5144e20cb124e3"
server_key_iv: "bf328d6458cb871d8cd78eddc49ce0f0"
server_key_version: "encryptionkey@0"
I20260809 08:56:00.875183 3493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:00.893131 3493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:00.895386 3493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:00.895565 3493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:00.895815 3493 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:56:00.896183 3493 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:56:00.896236 3493 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:00.896278 3493 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:56:00.896307 3493 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:00.902595 3493 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.1:38147
I20260809 08:56:00.902662 3693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.1:38147 every 8 connection(s)
I20260809 08:56:00.903029 3493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
I20260809 08:56:00.908275 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 3493
I20260809 08:56:00.908375 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance
I20260809 08:56:00.908576 8808 external_mini_cluster.cc:1428] Setting key 2ee8ef33b9ff65a7e67b6ec8269b0ec9
Aug 09 08:56:00 dist-test-slave-t3q3 krb5kdc[3332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265760, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:56:00.913125 3697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:00.903769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:35361 (local address 127.8.154.1:41969)
0809 08:56:00.903938 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:00.903977 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:00.904505 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:00.904773 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:00.904776 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:00.905353 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:56:00.905532 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:00.905538 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:00.906248 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:00.906249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:00.907068 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:00.907070 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:00.907474 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:00.907476 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:00.907600 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:00.908103 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:56:00.908111 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:56:00.909896 (+ 1785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:56:00.911779 (+ 1883us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:00.911785 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:00.911787 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:00.911999 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:00.912248 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:00.912249 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:00.912249 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:00.912350 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:00.912641 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:56:00.912646 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:56:00.912758 (+ 112us) client_negotiation.cc:769] Sending connection context
0809 08:56:00.912840 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 08:56:00.912956 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":89,"threads_started":1}
I20260809 08:56:00.913141 3489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:00.903796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:41969 (local address 127.8.154.62:35361)
0809 08:56:00.903865 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:00.903868 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:00.904084 (+ 216us) server_negotiation.cc:407] Connection header received
0809 08:56:00.904622 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:00.904623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:00.904641 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:00.904698 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:00.905645 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:00.906146 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:00.907179 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:00.907391 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:00.907466 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:00.909986 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:00.909988 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:00.909988 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:00.909994 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:00.911693 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:00.912102 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:00.912103 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:00.912104 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:00.912177 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:00.912418 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:00.912419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:00.912419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:00.912572 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:00.912655 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:00.912886 (+ 231us) server_negotiation.cc:299] Negotiation successful
0809 08:56:00.913003 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 08:56:00.913570 3694 heartbeater.cc:347] Connected to a master server at 127.8.154.62:35361
I20260809 08:56:00.913687 3694 heartbeater.cc:464] Registering TS with master...
I20260809 08:56:00.913897 3694 heartbeater.cc:511] Master 127.8.154.62:35361 requested a full tablet report, sending...
I20260809 08:56:00.914527 3399 ts_manager.cc:194] Registered new tserver with Master: 32f61fade8324186a304c52f09b3455a (127.8.154.1:38147)
I20260809 08:56:00.915210 3399 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.1@KRBTEST.COM'} at 127.8.154.1:41969
WARNING: no policy specified for kudu/127.8.154.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab.
Entry for principal kudu/127.8.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab.
I20260809 08:56:00.936095 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.2:0
--local_ip_for_outbound_sockets=127.8.154.2
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:35361
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:39077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:56:01.018203 3700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:01.018436 3700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:01.018472 3700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:01.019603 3700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:01.019667 3700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:01.019711 3700 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.2
I20260809 08:56:01.021337 3700 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:39077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:35361
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3700
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.2
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:01.021584 3700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:01.021840 3700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:01.026993 3717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:01.027047 3714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:01.027057 3715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:01.028178 3700 server_base.cc:1034] running on GCE node
I20260809 08:56:01.028435 3700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:01.028710 3700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:01.029879 3700 hybrid_clock.cc:648] HybridClock initialized: now 1786265761029859 us; error 36 us; skew 500 ppm
Aug 09 08:56:01 dist-test-slave-t3q3 krb5kdc[3332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265761, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:56:01.032395 3700 init.cc:377] Logged in from keytab as kudu/127.8.154.2@KRBTEST.COM (short username kudu)
I20260809 08:56:01.032868 3700 webserver.cc:460] Webserver started at http://127.8.154.2:42287/ using document root <none> and password file <none>
I20260809 08:56:01.033141 3700 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:01.033190 3700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:01.033335 3700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:01.034394 3700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-1/data/instance:
uuid: "e55e3dbd71864c1680bbbbda21828c00"
format_stamp: "Formatted at 2026-08-09 08:56:01 on dist-test-slave-t3q3"
server_key: "9d79cd84040edafa9c8d16276638ceb9"
server_key_iv: "fc40d7f4cef162b058407da7ee7b918f"
server_key_version: "encryptionkey@0"
I20260809 08:56:01.034765 3700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance:
uuid: "e55e3dbd71864c1680bbbbda21828c00"
format_stamp: "Formatted at 2026-08-09 08:56:01 on dist-test-slave-t3q3"
server_key: "9d79cd84040edafa9c8d16276638ceb9"
server_key_iv: "fc40d7f4cef162b058407da7ee7b918f"
server_key_version: "encryptionkey@0"
I20260809 08:56:01.036175 3700 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:56:01.036888 3726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:01.037058 3700 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:56:01.037166 3700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-1/wal
uuid: "e55e3dbd71864c1680bbbbda21828c00"
format_stamp: "Formatted at 2026-08-09 08:56:01 on dist-test-slave-t3q3"
server_key: "9d79cd84040edafa9c8d16276638ceb9"
server_key_iv: "fc40d7f4cef162b058407da7ee7b918f"
server_key_version: "encryptionkey@0"
I20260809 08:56:01.037245 3700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:01.056792 3700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:01.058643 3700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:01.058808 3700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:01.059070 3700 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:56:01.059444 3700 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:56:01.059481 3700 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:01.059504 3700 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:56:01.059517 3700 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:01.065608 3700 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.2:34575
I20260809 08:56:01.065663 3891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.2:34575 every 8 connection(s)
I20260809 08:56:01.065963 3700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
I20260809 08:56:01.070914 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 3700
I20260809 08:56:01.071031 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance
I20260809 08:56:01.071233 8808 external_mini_cluster.cc:1428] Setting key b753e7ae2e24f0d0b6a73c0d4c12e493
Aug 09 08:56:01 dist-test-slave-t3q3 krb5kdc[3332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265761, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:56:01.076017 3896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:01.066612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:35361 (local address 127.8.154.2:48589)
0809 08:56:01.066806 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:01.066842 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:01.067350 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:01.067604 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:01.067607 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:01.068260 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:56:01.068458 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:01.068463 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.069168 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.069169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:01.069977 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:01.069979 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.070374 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.070375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:01.070513 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:01.071034 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:56:01.071046 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:56:01.072944 (+ 1898us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:56:01.074579 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:01.074586 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:01.074589 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:01.074911 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:01.075096 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:01.075097 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:01.075097 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:01.075241 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:01.075534 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:56:01.075539 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:56:01.075658 (+ 119us) client_negotiation.cc:769] Sending connection context
0809 08:56:01.075741 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 08:56:01.075862 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":99,"threads_started":1}
I20260809 08:56:01.076027 3489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:01.066658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:48589 (local address 127.8.154.62:35361)
0809 08:56:01.066737 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:01.066740 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:01.066928 (+ 188us) server_negotiation.cc:407] Connection header received
0809 08:56:01.067457 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:01.067458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:01.067471 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:01.067520 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:01.068563 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.069072 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.070074 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.070292 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.070368 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:01.073045 (+ 2677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:01.073046 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:01.073046 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:01.073052 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:01.074498 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:01.074986 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:01.074987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:01.074987 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:01.075037 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:01.075314 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:01.075315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:01.075315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:01.075465 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:01.075540 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:01.075786 (+ 246us) server_negotiation.cc:299] Negotiation successful
0809 08:56:01.075885 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 08:56:01.076650 3893 heartbeater.cc:347] Connected to a master server at 127.8.154.62:35361
I20260809 08:56:01.076758 3893 heartbeater.cc:464] Registering TS with master...
I20260809 08:56:01.076974 3893 heartbeater.cc:511] Master 127.8.154.62:35361 requested a full tablet report, sending...
I20260809 08:56:01.077370 3399 ts_manager.cc:194] Registered new tserver with Master: e55e3dbd71864c1680bbbbda21828c00 (127.8.154.2:34575)
I20260809 08:56:01.077730 3399 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.2@KRBTEST.COM'} at 127.8.154.2:48589
WARNING: no policy specified for kudu/127.8.154.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab.
Entry for principal kudu/127.8.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab.
I20260809 08:56:01.098837 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.3:0
--local_ip_for_outbound_sockets=127.8.154.3
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:35361
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:39077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:56:01.182547 3900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:01.182762 3900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:01.182786 3900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:01.183954 3900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:01.184007 3900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:01.184036 3900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.3
I20260809 08:56:01.185551 3900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:39077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:35361
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.3
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:01.185822 3900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:01.186064 3900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 08:56:01.191473 3900 server_base.cc:1034] running on GCE node
W20260809 08:56:01.191447 3912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:01.191442 3916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:01.191563 3911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:01.191944 3900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:01.192234 3900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:01.193388 3900 hybrid_clock.cc:648] HybridClock initialized: now 1786265761193375 us; error 30 us; skew 500 ppm
Aug 09 08:56:01 dist-test-slave-t3q3 krb5kdc[3332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265761, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:56:01.196095 3900 init.cc:377] Logged in from keytab as kudu/127.8.154.3@KRBTEST.COM (short username kudu)
I20260809 08:56:01.196542 3900 webserver.cc:460] Webserver started at http://127.8.154.3:39869/ using document root <none> and password file <none>
I20260809 08:56:01.196771 3900 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:01.196835 3900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:01.196983 3900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:01.198133 3900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-2/data/instance:
uuid: "85b6199de7e8422e88a4760af8b81564"
format_stamp: "Formatted at 2026-08-09 08:56:01 on dist-test-slave-t3q3"
server_key: "8bb03a09e400dbcd26b2e411b8bac1b1"
server_key_iv: "0b8a7524397c2d240e7cd786058d6877"
server_key_version: "encryptionkey@0"
I20260809 08:56:01.198518 3900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance:
uuid: "85b6199de7e8422e88a4760af8b81564"
format_stamp: "Formatted at 2026-08-09 08:56:01 on dist-test-slave-t3q3"
server_key: "8bb03a09e400dbcd26b2e411b8bac1b1"
server_key_iv: "0b8a7524397c2d240e7cd786058d6877"
server_key_version: "encryptionkey@0"
I20260809 08:56:01.199942 3900 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:56:01.200821 3925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:01.200982 3900 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:56:01.201083 3900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-2/wal
uuid: "85b6199de7e8422e88a4760af8b81564"
format_stamp: "Formatted at 2026-08-09 08:56:01 on dist-test-slave-t3q3"
server_key: "8bb03a09e400dbcd26b2e411b8bac1b1"
server_key_iv: "0b8a7524397c2d240e7cd786058d6877"
server_key_version: "encryptionkey@0"
I20260809 08:56:01.201160 3900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:01.217689 3900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:01.220224 3900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:01.220422 3900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:01.220695 3900 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:56:01.221035 3900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:56:01.221087 3900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:01.221132 3900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:56:01.221158 3900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:01.227557 3900 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.3:36703
I20260809 08:56:01.227608 4100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.3:36703 every 8 connection(s)
I20260809 08:56:01.227977 3900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
I20260809 08:56:01.234488 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 3900
I20260809 08:56:01.234598 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance
Aug 09 08:56:01 dist-test-slave-t3q3 krb5kdc[3332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265761, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:56:01.234824 8808 external_mini_cluster.cc:1428] Setting key a19a1023ce2af1e70c98ce3b9290eb9b
I20260809 08:56:01.238062 3489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:01.228758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:46573 (local address 127.8.154.62:35361)
0809 08:56:01.228804 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:01.228806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:01.229064 (+ 258us) server_negotiation.cc:407] Connection header received
0809 08:56:01.229625 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:01.229626 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:01.229639 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:01.229703 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:01.230704 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.231215 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.232343 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.232540 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.232630 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:01.235303 (+ 2673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:01.235304 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:01.235304 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:01.235311 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:01.236670 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:01.237071 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:01.237072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:01.237072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:01.237138 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:01.237389 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:01.237390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:01.237390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:01.237512 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:01.237583 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:01.237883 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 08:56:01.237970 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:56:01.238260 4103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:01.228762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:35361 (local address 127.8.154.3:46573)
0809 08:56:01.228935 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:01.228972 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:01.229509 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:01.229779 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:01.229783 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:01.230391 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:56:01.230599 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:01.230604 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.231315 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.231316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:01.232253 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:01.232255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.232640 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.232642 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:01.232777 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:01.233413 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:56:01.233422 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:56:01.235152 (+ 1730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:56:01.236750 (+ 1598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:01.236759 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:01.236761 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:01.236994 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:01.237205 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:01.237206 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:01.237206 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:01.237323 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:01.237597 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:56:01.237605 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:56:01.237743 (+ 138us) client_negotiation.cc:769] Sending connection context
0809 08:56:01.237846 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 08:56:01.238018 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":90,"threads_started":1}
I20260809 08:56:01.238741 4101 heartbeater.cc:347] Connected to a master server at 127.8.154.62:35361
I20260809 08:56:01.238850 4101 heartbeater.cc:464] Registering TS with master...
I20260809 08:56:01.239055 4101 heartbeater.cc:511] Master 127.8.154.62:35361 requested a full tablet report, sending...
I20260809 08:56:01.239465 3399 ts_manager.cc:194] Registered new tserver with Master: 85b6199de7e8422e88a4760af8b81564 (127.8.154.3:36703)
I20260809 08:56:01.239835 3399 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.3@KRBTEST.COM'} at 127.8.154.3:46573
I20260809 08:56:01.246563 8808 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:56:01.254027 3489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:01.247904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42380 (local address 127.8.154.62:35361)
0809 08:56:01.247971 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:01.247974 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:01.248116 (+ 142us) server_negotiation.cc:407] Connection header received
0809 08:56:01.248195 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:01.248196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:01.248206 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:01.248258 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:01.248857 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.249441 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.250220 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.250409 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.250477 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:01.251699 (+ 1222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:01.251700 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:01.251700 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:01.251706 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:01.253031 (+ 1325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:01.253349 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:01.253350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:01.253350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:01.253404 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:01.253567 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:01.253567 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:01.253567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:01.253718 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:01.253794 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:01.253879 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 08:56:01.253937 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 08:56:01.255652 3399 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42380:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:56:01.257340 3399 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:56:01.264007 4118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:01.259802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:34575 (local address 127.0.0.1:47894)
0809 08:56:01.260187 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:01.260216 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:01.260274 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:01.260590 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:01.260593 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:01.260594 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:56:01.260670 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:01.260672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.262063 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.262064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:01.262992 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:01.262995 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.263693 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.263694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:01.263784 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:01.263798 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 08:56:01.263870 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 08:56:01.263923 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":98,"threads_started":1}
I20260809 08:56:01.264271 4120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:01.259916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47894 (local address 127.8.154.2:34575)
0809 08:56:01.260142 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:01.260147 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:01.260361 (+ 214us) server_negotiation.cc:407] Connection header received
0809 08:56:01.260409 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:01.260412 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:01.260429 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:01.260501 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:56:01.260739 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.261926 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.263076 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.263608 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.263692 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:01.263757 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:01.264156 (+ 399us) server_negotiation.cc:299] Negotiation successful
0809 08:56:01.264197 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":83,"threads_started":1}
I20260809 08:56:01.264305 4123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:01.260139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33990 (local address 127.8.154.1:38147)
0809 08:56:01.260301 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:01.260306 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:01.260490 (+ 184us) server_negotiation.cc:407] Connection header received
0809 08:56:01.260536 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:01.260538 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:01.260554 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:01.260625 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:56:01.260875 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.261589 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.263003 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.263774 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.263852 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:01.263912 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:01.263988 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 08:56:01.264035 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":89,"threads_started":1}
I20260809 08:56:01.264302 4117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:01.260006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:38147 (local address 127.0.0.1:33990)
0809 08:56:01.260373 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:01.260383 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:01.260425 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:01.260722 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:01.260723 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:01.260725 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:56:01.260789 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:01.260791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.261706 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.261707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:01.262910 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:01.262913 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.263851 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.263852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:01.263909 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:01.263912 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:56:01.263965 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 08:56:01.264000 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"mutex_wait_us":21,"thread_start_us":88,"threads_started":1}
I20260809 08:56:01.264827 4119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:01.259883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52372 (local address 127.8.154.3:36703)
0809 08:56:01.260375 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:01.260379 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:01.260391 (+ 12us) server_negotiation.cc:407] Connection header received
0809 08:56:01.260508 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:01.260511 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:01.260538 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:01.260623 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:56:01.260886 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.261943 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.263197 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.263894 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.263960 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:01.264016 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:01.264603 (+ 587us) server_negotiation.cc:299] Negotiation successful
0809 08:56:01.264760 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":115,"threads_started":1}
I20260809 08:56:01.264545 4122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:01.259787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:36703 (local address 127.0.0.1:52372)
0809 08:56:01.260158 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:01.260204 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:01.260274 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:01.260713 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:01.260714 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:01.260716 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:56:01.260798 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:01.260800 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.262045 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.262046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:01.263119 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:01.263121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.264199 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.264200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:01.264268 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:01.264272 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:56:01.264457 (+ 185us) client_negotiation.cc:240] Negotiation successful
0809 08:56:01.264493 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":30,"thread_start_us":47,"threads_started":1}
I20260809 08:56:01.265030 3790 tablet_service.cc:1467] Processing CreateTablet for tablet de52eaa254564148936da4c40f2b9aea (DEFAULT_TABLE table=test-table [id=b9ceca03378d402e8f0a30176379f790]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:56:01.265399 3790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de52eaa254564148936da4c40f2b9aea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:01.265508 3595 tablet_service.cc:1467] Processing CreateTablet for tablet de52eaa254564148936da4c40f2b9aea (DEFAULT_TABLE table=test-table [id=b9ceca03378d402e8f0a30176379f790]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:56:01.265851 3595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de52eaa254564148936da4c40f2b9aea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:01.265826 3998 tablet_service.cc:1467] Processing CreateTablet for tablet de52eaa254564148936da4c40f2b9aea (DEFAULT_TABLE table=test-table [id=b9ceca03378d402e8f0a30176379f790]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:56:01.266134 3998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de52eaa254564148936da4c40f2b9aea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:01.267997 4124 tablet_bootstrap.cc:492] T de52eaa254564148936da4c40f2b9aea P e55e3dbd71864c1680bbbbda21828c00: Bootstrap starting.
I20260809 08:56:01.268731 4124 tablet_bootstrap.cc:654] T de52eaa254564148936da4c40f2b9aea P e55e3dbd71864c1680bbbbda21828c00: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:01.269223 4124 log.cc:824] T de52eaa254564148936da4c40f2b9aea P e55e3dbd71864c1680bbbbda21828c00: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:01.269474 4126 tablet_bootstrap.cc:492] T de52eaa254564148936da4c40f2b9aea P 32f61fade8324186a304c52f09b3455a: Bootstrap starting.
I20260809 08:56:01.270131 4126 tablet_bootstrap.cc:654] T de52eaa254564148936da4c40f2b9aea P 32f61fade8324186a304c52f09b3455a: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:01.270227 4124 tablet_bootstrap.cc:492] T de52eaa254564148936da4c40f2b9aea P e55e3dbd71864c1680bbbbda21828c00: No bootstrap required, opened a new log
I20260809 08:56:01.270308 4124 ts_tablet_manager.cc:1397] T de52eaa254564148936da4c40f2b9aea P e55e3dbd71864c1680bbbbda21828c00: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 08:56:01.270751 4126 log.cc:824] T de52eaa254564148936da4c40f2b9aea P 32f61fade8324186a304c52f09b3455a: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:01.271098 4125 tablet_bootstrap.cc:492] T de52eaa254564148936da4c40f2b9aea P 85b6199de7e8422e88a4760af8b81564: Bootstrap starting.
I20260809 08:56:01.271450 4126 tablet_bootstrap.cc:492] T de52eaa254564148936da4c40f2b9aea P 32f61fade8324186a304c52f09b3455a: No bootstrap required, opened a new log
I20260809 08:56:01.271538 4126 ts_tablet_manager.cc:1397] T de52eaa254564148936da4c40f2b9aea P 32f61fade8324186a304c52f09b3455a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:56:01.271906 4125 tablet_bootstrap.cc:654] T de52eaa254564148936da4c40f2b9aea P 85b6199de7e8422e88a4760af8b81564: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:01.272495 4125 log.cc:824] T de52eaa254564148936da4c40f2b9aea P 85b6199de7e8422e88a4760af8b81564: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:01.272686 4124 raft_consensus.cc:348] T de52eaa254564148936da4c40f2b9aea P e55e3dbd71864c1680bbbbda21828c00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85b6199de7e8422e88a4760af8b81564" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 36703 } } peers { permanent_uuid: "e55e3dbd71864c1680bbbbda21828c00" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 34575 } } peers { permanent_uuid: "32f61fade8324186a304c52f09b3455a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 38147 } }
I20260809 08:56:01.272886 4124 raft_consensus.cc:374] T de52eaa254564148936da4c40f2b9aea P e55e3dbd71864c1680bbbbda21828c00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:01.272934 4124 raft_consensus.cc:729] T de52eaa254564148936da4c40f2b9aea P e55e3dbd71864c1680bbbbda21828c00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e55e3dbd71864c1680bbbbda21828c00, State: Initialized, Role: FOLLOWER
I20260809 08:56:01.273063 4124 consensus_queue.cc:260] T de52eaa254564148936da4c40f2b9aea P e55e3dbd71864c1680bbbbda21828c00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85b6199de7e8422e88a4760af8b81564" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 36703 } } peers { permanent_uuid: "e55e3dbd71864c1680bbbbda21828c00" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 34575 } } peers { permanent_uuid: "32f61fade8324186a304c52f09b3455a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 38147 } }
I20260809 08:56:01.273387 4124 ts_tablet_manager.cc:1428] T de52eaa254564148936da4c40f2b9aea P e55e3dbd71864c1680bbbbda21828c00: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 08:56:01.273408 4125 tablet_bootstrap.cc:492] T de52eaa254564148936da4c40f2b9aea P 85b6199de7e8422e88a4760af8b81564: No bootstrap required, opened a new log
I20260809 08:56:01.273465 4125 ts_tablet_manager.cc:1397] T de52eaa254564148936da4c40f2b9aea P 85b6199de7e8422e88a4760af8b81564: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260809 08:56:01.273520 3893 heartbeater.cc:503] Master 127.8.154.62:35361 was elected leader, sending a full tablet report...
I20260809 08:56:01.273516 4126 raft_consensus.cc:348] T de52eaa254564148936da4c40f2b9aea P 32f61fade8324186a304c52f09b3455a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85b6199de7e8422e88a4760af8b81564" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 36703 } } peers { permanent_uuid: "e55e3dbd71864c1680bbbbda21828c00" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 34575 } } peers { permanent_uuid: "32f61fade8324186a304c52f09b3455a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 38147 } }
I20260809 08:56:01.273710 4126 raft_consensus.cc:374] T de52eaa254564148936da4c40f2b9aea P 32f61fade8324186a304c52f09b3455a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:01.273762 4126 raft_consensus.cc:729] T de52eaa254564148936da4c40f2b9aea P 32f61fade8324186a304c52f09b3455a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32f61fade8324186a304c52f09b3455a, State: Initialized, Role: FOLLOWER
I20260809 08:56:01.273880 4126 consensus_queue.cc:260] T de52eaa254564148936da4c40f2b9aea P 32f61fade8324186a304c52f09b3455a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85b6199de7e8422e88a4760af8b81564" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 36703 } } peers { permanent_uuid: "e55e3dbd71864c1680bbbbda21828c00" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 34575 } } peers { permanent_uuid: "32f61fade8324186a304c52f09b3455a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 38147 } }
I20260809 08:56:01.274160 4126 ts_tablet_manager.cc:1428] T de52eaa254564148936da4c40f2b9aea P 32f61fade8324186a304c52f09b3455a: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 08:56:01.274186 3694 heartbeater.cc:503] Master 127.8.154.62:35361 was elected leader, sending a full tablet report...
I20260809 08:56:01.275544 4125 raft_consensus.cc:348] T de52eaa254564148936da4c40f2b9aea P 85b6199de7e8422e88a4760af8b81564 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85b6199de7e8422e88a4760af8b81564" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 36703 } } peers { permanent_uuid: "e55e3dbd71864c1680bbbbda21828c00" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 34575 } } peers { permanent_uuid: "32f61fade8324186a304c52f09b3455a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 38147 } }
I20260809 08:56:01.275671 4125 raft_consensus.cc:374] T de52eaa254564148936da4c40f2b9aea P 85b6199de7e8422e88a4760af8b81564 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:01.275691 4125 raft_consensus.cc:729] T de52eaa254564148936da4c40f2b9aea P 85b6199de7e8422e88a4760af8b81564 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85b6199de7e8422e88a4760af8b81564, State: Initialized, Role: FOLLOWER
I20260809 08:56:01.275794 4125 consensus_queue.cc:260] T de52eaa254564148936da4c40f2b9aea P 85b6199de7e8422e88a4760af8b81564 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85b6199de7e8422e88a4760af8b81564" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 36703 } } peers { permanent_uuid: "e55e3dbd71864c1680bbbbda21828c00" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 34575 } } peers { permanent_uuid: "32f61fade8324186a304c52f09b3455a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 38147 } }
I20260809 08:56:01.276132 4125 ts_tablet_manager.cc:1428] T de52eaa254564148936da4c40f2b9aea P 85b6199de7e8422e88a4760af8b81564: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 08:56:01.276170 4101 heartbeater.cc:503] Master 127.8.154.62:35361 was elected leader, sending a full tablet report...
W20260809 08:56:01.316509 3894 tablet.cc:2369] T de52eaa254564148936da4c40f2b9aea P e55e3dbd71864c1680bbbbda21828c00: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:56:01.364377 4135 raft_consensus.cc:482] T de52eaa254564148936da4c40f2b9aea P 85b6199de7e8422e88a4760af8b81564 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:56:01.364509 4135 raft_consensus.cc:504] T de52eaa254564148936da4c40f2b9aea P 85b6199de7e8422e88a4760af8b81564 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85b6199de7e8422e88a4760af8b81564" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 36703 } } peers { permanent_uuid: "e55e3dbd71864c1680bbbbda21828c00" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 34575 } } peers { permanent_uuid: "32f61fade8324186a304c52f09b3455a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 38147 } }
I20260809 08:56:01.364871 4135 leader_election.cc:288] T de52eaa254564148936da4c40f2b9aea P 85b6199de7e8422e88a4760af8b81564 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e55e3dbd71864c1680bbbbda21828c00 (127.8.154.2:34575), 32f61fade8324186a304c52f09b3455a (127.8.154.1:38147)
I20260809 08:56:01.367779 4120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:01.365178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:43679 (local address 127.8.154.2:34575)
0809 08:56:01.365233 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:01.365235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:01.365247 (+ 12us) server_negotiation.cc:407] Connection header received
0809 08:56:01.365284 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:01.365285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:01.365298 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:01.365356 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:56:01.365536 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.366020 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.367034 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.367423 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.367510 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:01.367522 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:01.367679 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 08:56:01.367718 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 08:56:01.367779 4103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:01.365065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:34575 (local address 127.8.154.3:43679)
0809 08:56:01.365137 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:01.365152 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:01.365231 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:01.365387 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:01.365389 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:01.365391 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:56:01.365460 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:01.365462 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.366124 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.366125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:01.366936 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:01.366939 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.367509 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.367510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:01.367592 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:01.367595 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:56:01.367658 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 08:56:01.367701 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":5}
I20260809 08:56:01.367991 4136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:01.365192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:38147 (local address 127.8.154.3:34047)
0809 08:56:01.365408 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:01.365424 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:01.365490 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:01.365639 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:01.365640 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:01.365641 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:56:01.365717 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:01.365718 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.366358 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.366359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:01.367128 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:01.367130 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.367697 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.367698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:01.367770 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:01.367779 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 08:56:01.367841 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 08:56:01.367887 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":135,"threads_started":1}
I20260809 08:56:01.368207 4123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:01.365301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:34047 (local address 127.8.154.1:38147)
0809 08:56:01.365365 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:01.365367 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:01.365482 (+ 115us) server_negotiation.cc:407] Connection header received
0809 08:56:01.365549 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:01.365550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:01.365563 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:01.365634 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:56:01.365792 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.366258 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.367221 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.367612 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.367693 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:01.367703 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:01.367974 (+ 271us) server_negotiation.cc:299] Negotiation successful
0809 08:56:01.368021 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 08:56:01.368196 3824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de52eaa254564148936da4c40f2b9aea" candidate_uuid: "85b6199de7e8422e88a4760af8b81564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e55e3dbd71864c1680bbbbda21828c00" is_pre_election: true
I20260809 08:56:01.368351 3824 raft_consensus.cc:2454] T de52eaa254564148936da4c40f2b9aea P e55e3dbd71864c1680bbbbda21828c00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85b6199de7e8422e88a4760af8b81564 in term 0.
I20260809 08:56:01.368649 3932 leader_election.cc:302] T de52eaa254564148936da4c40f2b9aea P 85b6199de7e8422e88a4760af8b81564 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85b6199de7e8422e88a4760af8b81564, e55e3dbd71864c1680bbbbda21828c00; no voters:
I20260809 08:56:01.368818 4135 raft_consensus.cc:2781] T de52eaa254564148936da4c40f2b9aea P 85b6199de7e8422e88a4760af8b81564 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:56:01.368785 3623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de52eaa254564148936da4c40f2b9aea" candidate_uuid: "85b6199de7e8422e88a4760af8b81564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32f61fade8324186a304c52f09b3455a" is_pre_election: true
I20260809 08:56:01.368909 4135 raft_consensus.cc:482] T de52eaa254564148936da4c40f2b9aea P 85b6199de7e8422e88a4760af8b81564 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:56:01.368945 4135 raft_consensus.cc:3037] T de52eaa254564148936da4c40f2b9aea P 85b6199de7e8422e88a4760af8b81564 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:01.368981 3623 raft_consensus.cc:2454] T de52eaa254564148936da4c40f2b9aea P 32f61fade8324186a304c52f09b3455a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 85b6199de7e8422e88a4760af8b81564 in term 0.
I20260809 08:56:01.369731 4135 raft_consensus.cc:504] T de52eaa254564148936da4c40f2b9aea P 85b6199de7e8422e88a4760af8b81564 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85b6199de7e8422e88a4760af8b81564" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 36703 } } peers { permanent_uuid: "e55e3dbd71864c1680bbbbda21828c00" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 34575 } } peers { permanent_uuid: "32f61fade8324186a304c52f09b3455a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 38147 } }
I20260809 08:56:01.369871 4135 leader_election.cc:288] T de52eaa254564148936da4c40f2b9aea P 85b6199de7e8422e88a4760af8b81564 [CANDIDATE]: Term 1 election: Requested vote from peers e55e3dbd71864c1680bbbbda21828c00 (127.8.154.2:34575), 32f61fade8324186a304c52f09b3455a (127.8.154.1:38147)
I20260809 08:56:01.370046 3824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de52eaa254564148936da4c40f2b9aea" candidate_uuid: "85b6199de7e8422e88a4760af8b81564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e55e3dbd71864c1680bbbbda21828c00"
I20260809 08:56:01.370069 3623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de52eaa254564148936da4c40f2b9aea" candidate_uuid: "85b6199de7e8422e88a4760af8b81564" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "32f61fade8324186a304c52f09b3455a"
I20260809 08:56:01.370143 3824 raft_consensus.cc:3037] T de52eaa254564148936da4c40f2b9aea P e55e3dbd71864c1680bbbbda21828c00 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:01.370137 3623 raft_consensus.cc:3037] T de52eaa254564148936da4c40f2b9aea P 32f61fade8324186a304c52f09b3455a [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:01.370895 3623 raft_consensus.cc:2454] T de52eaa254564148936da4c40f2b9aea P 32f61fade8324186a304c52f09b3455a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85b6199de7e8422e88a4760af8b81564 in term 1.
I20260809 08:56:01.370899 3824 raft_consensus.cc:2454] T de52eaa254564148936da4c40f2b9aea P e55e3dbd71864c1680bbbbda21828c00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85b6199de7e8422e88a4760af8b81564 in term 1.
I20260809 08:56:01.371081 3932 leader_election.cc:302] T de52eaa254564148936da4c40f2b9aea P 85b6199de7e8422e88a4760af8b81564 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32f61fade8324186a304c52f09b3455a, 85b6199de7e8422e88a4760af8b81564; no voters:
I20260809 08:56:01.371196 4135 raft_consensus.cc:2781] T de52eaa254564148936da4c40f2b9aea P 85b6199de7e8422e88a4760af8b81564 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:56:01.371344 4135 raft_consensus.cc:686] T de52eaa254564148936da4c40f2b9aea P 85b6199de7e8422e88a4760af8b81564 [term 1 LEADER]: Becoming Leader. State: Replica: 85b6199de7e8422e88a4760af8b81564, State: Running, Role: LEADER
I20260809 08:56:01.371557 4135 consensus_queue.cc:237] T de52eaa254564148936da4c40f2b9aea P 85b6199de7e8422e88a4760af8b81564 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85b6199de7e8422e88a4760af8b81564" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 36703 } } peers { permanent_uuid: "e55e3dbd71864c1680bbbbda21828c00" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 34575 } } peers { permanent_uuid: "32f61fade8324186a304c52f09b3455a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 38147 } }
I20260809 08:56:01.372506 3399 catalog_manager.cc:5495] T de52eaa254564148936da4c40f2b9aea P 85b6199de7e8422e88a4760af8b81564 reported cstate change: term changed from 0 to 1, leader changed from <none> to 85b6199de7e8422e88a4760af8b81564 (127.8.154.3). New cstate: current_term: 1 leader_uuid: "85b6199de7e8422e88a4760af8b81564" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85b6199de7e8422e88a4760af8b81564" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 36703 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e55e3dbd71864c1680bbbbda21828c00" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 34575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32f61fade8324186a304c52f09b3455a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 38147 } health_report { overall_health: UNKNOWN } } }
I20260809 08:56:01.378706 4119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:01.376553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52388 (local address 127.8.154.3:36703)
0809 08:56:01.376592 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:01.376594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:01.376601 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:56:01.376681 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:01.376681 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:01.376697 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:01.376748 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:56:01.376836 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.377273 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.377967 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.378205 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.378283 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:01.378426 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:56:01.378521 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:56:01.378590 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:01.378625 (+ 35us) server_negotiation.cc:299] Negotiation successful
0809 08:56:01.378644 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:56:01.380352 3623 raft_consensus.cc:1261] T de52eaa254564148936da4c40f2b9aea P 32f61fade8324186a304c52f09b3455a [term 1 FOLLOWER]: Refusing update from remote peer 85b6199de7e8422e88a4760af8b81564: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:56:01.380357 3824 raft_consensus.cc:1261] T de52eaa254564148936da4c40f2b9aea P e55e3dbd71864c1680bbbbda21828c00 [term 1 FOLLOWER]: Refusing update from remote peer 85b6199de7e8422e88a4760af8b81564: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:56:01.380676 4135 consensus_queue.cc:1035] T de52eaa254564148936da4c40f2b9aea P 85b6199de7e8422e88a4760af8b81564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "32f61fade8324186a304c52f09b3455a" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 38147 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:56:01.380798 4137 consensus_queue.cc:1035] T de52eaa254564148936da4c40f2b9aea P 85b6199de7e8422e88a4760af8b81564 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e55e3dbd71864c1680bbbbda21828c00" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 34575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:56:01.383808 4144 mvcc.cc:205] Tried to move back new op lower bound from 7316544558611542016 to 7316544558578663424. Current Snapshot: MvccSnapshot[applied={T|T < 7316544558611542016 or (T in {7316544558611542016})}]
I20260809 08:56:01.385370 3395 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42380:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:56:01.385473 3395 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42380:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:56:01.386217 3395 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 257ff16d7c524971a65bf1860994860f: Sending DeleteTablet for 3 replicas of tablet de52eaa254564148936da4c40f2b9aea
I20260809 08:56:01.386562 3790 tablet_service.cc:1514] Processing DeleteTablet for tablet de52eaa254564148936da4c40f2b9aea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:56:01 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:47894
I20260809 08:56:01.386610 3595 tablet_service.cc:1514] Processing DeleteTablet for tablet de52eaa254564148936da4c40f2b9aea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:56:01 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:33990
I20260809 08:56:01.386715 3998 tablet_service.cc:1514] Processing DeleteTablet for tablet de52eaa254564148936da4c40f2b9aea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:56:01 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:52372
I20260809 08:56:01.386937 4154 tablet_replica.cc:330] T de52eaa254564148936da4c40f2b9aea P e55e3dbd71864c1680bbbbda21828c00: stopping tablet replica
I20260809 08:56:01.387084 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 3493
I20260809 08:56:01.391446 4154 raft_consensus.cc:2229] T de52eaa254564148936da4c40f2b9aea P e55e3dbd71864c1680bbbbda21828c00 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:56:01.391582 4154 raft_consensus.cc:2258] T de52eaa254564148936da4c40f2b9aea P e55e3dbd71864c1680bbbbda21828c00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:56:01.393607 4154 ts_tablet_manager.cc:1905] T de52eaa254564148936da4c40f2b9aea P e55e3dbd71864c1680bbbbda21828c00: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:56:01.395044 4156 tablet_replica.cc:330] T de52eaa254564148936da4c40f2b9aea P 85b6199de7e8422e88a4760af8b81564: stopping tablet replica
I20260809 08:56:01.395169 4156 raft_consensus.cc:2229] T de52eaa254564148936da4c40f2b9aea P 85b6199de7e8422e88a4760af8b81564 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:56:01.395290 4156 raft_consensus.cc:2258] T de52eaa254564148936da4c40f2b9aea P 85b6199de7e8422e88a4760af8b81564 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:56:01.396675 4154 ts_tablet_manager.cc:1918] T de52eaa254564148936da4c40f2b9aea P e55e3dbd71864c1680bbbbda21828c00: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:56:01.396749 4154 log.cc:1196] T de52eaa254564148936da4c40f2b9aea P e55e3dbd71864c1680bbbbda21828c00: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-1/wal/wals/de52eaa254564148936da4c40f2b9aea
I20260809 08:56:01.397055 4154 ts_tablet_manager.cc:1939] T de52eaa254564148936da4c40f2b9aea P e55e3dbd71864c1680bbbbda21828c00: Deleting consensus metadata
I20260809 08:56:01.402099 4156 ts_tablet_manager.cc:1905] T de52eaa254564148936da4c40f2b9aea P 85b6199de7e8422e88a4760af8b81564: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:56:01.403148 3378 catalog_manager.cc:4841] TS e55e3dbd71864c1680bbbbda21828c00 (127.8.154.2:34575): tablet de52eaa254564148936da4c40f2b9aea (table test-table [id=b9ceca03378d402e8f0a30176379f790]) successfully deleted
I20260809 08:56:01.405030 4156 ts_tablet_manager.cc:1918] T de52eaa254564148936da4c40f2b9aea P 85b6199de7e8422e88a4760af8b81564: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:56:01.405128 4156 log.cc:1196] T de52eaa254564148936da4c40f2b9aea P 85b6199de7e8422e88a4760af8b81564: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265519942966-8808-0/minicluster-data/ts-2/wal/wals/de52eaa254564148936da4c40f2b9aea
I20260809 08:56:01.405390 4156 ts_tablet_manager.cc:1939] T de52eaa254564148936da4c40f2b9aea P 85b6199de7e8422e88a4760af8b81564: Deleting consensus metadata
I20260809 08:56:01.405850 3377 catalog_manager.cc:4841] TS 85b6199de7e8422e88a4760af8b81564 (127.8.154.3:36703): tablet de52eaa254564148936da4c40f2b9aea (table test-table [id=b9ceca03378d402e8f0a30176379f790]) successfully deleted
W20260809 08:56:01.412453 3378 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.154.1:38147 (error 108)
I20260809 08:56:01.412710 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 3700
I20260809 08:56:01.419562 4117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:01.419405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:38147 (local address 127.0.0.1:33992)
0809 08:56:01.419465 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:01.419515 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.1:38147: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 08:56:01.420861 3378 catalog_manager.cc:4584] TS 32f61fade8324186a304c52f09b3455a (127.8.154.1:38147): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet de52eaa254564148936da4c40f2b9aea: Network error: Client connection negotiation failed: client connection to 127.8.154.1:38147: connect: Connection refused (error 111)
I20260809 08:56:01.428567 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 3900
I20260809 08:56:01.439669 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 3348
2026-08-09T08:56:01Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2214 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T08:56:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:56:01Z Disabled control of system clock
I20260809 08:56:01.473455 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:44249
--webserver_interface=127.8.154.62
--webserver_port=0
--builtin_ntp_servers=127.8.154.20:36389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:44249
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 08:56:01.555106 4164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:01.555323 4164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:01.555344 4164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:01.556491 4164 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:56:01.556531 4164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:01.556545 4164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:01.556557 4164 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:56:01.556569 4164 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:56:01.557853 4164 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:36389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:44249
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:44249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4164
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:01.558048 4164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:01.558284 4164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:01.563150 4181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:01.563228 4174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:01.563347 4164 server_base.cc:1034] running on GCE node
W20260809 08:56:01.563320 4177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:01.563694 4164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:01.563982 4164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:01.565131 4164 hybrid_clock.cc:648] HybridClock initialized: now 1786265761565122 us; error 33 us; skew 500 ppm
I20260809 08:56:01.567663 4164 webserver.cc:460] Webserver started at http://127.8.154.62:34291/ using document root <none> and password file <none>
I20260809 08:56:01.567865 4164 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:01.567907 4164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:01.567996 4164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:01.569113 4164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "53e9822cd6054a2bbf0c5e5409e1fe2f"
format_stamp: "Formatted at 2026-08-09 08:56:01 on dist-test-slave-t3q3"
server_key: "6bb3e8b1ad0769ac2cd12146b9c30015"
server_key_iv: "90ccd052688365cea4643ee6edaab1be"
server_key_version: "encryptionkey@0"
I20260809 08:56:01.569435 4164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "53e9822cd6054a2bbf0c5e5409e1fe2f"
format_stamp: "Formatted at 2026-08-09 08:56:01 on dist-test-slave-t3q3"
server_key: "6bb3e8b1ad0769ac2cd12146b9c30015"
server_key_iv: "90ccd052688365cea4643ee6edaab1be"
server_key_version: "encryptionkey@0"
I20260809 08:56:01.570778 4164 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:56:01.571508 4189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:01.571689 4164 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:56:01.571766 4164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "53e9822cd6054a2bbf0c5e5409e1fe2f"
format_stamp: "Formatted at 2026-08-09 08:56:01 on dist-test-slave-t3q3"
server_key: "6bb3e8b1ad0769ac2cd12146b9c30015"
server_key_iv: "90ccd052688365cea4643ee6edaab1be"
server_key_version: "encryptionkey@0"
I20260809 08:56:01.571820 4164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:01.580461 4164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:01.580720 4164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:01.580824 4164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:01.586028 4164 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:44249
I20260809 08:56:01.586064 4274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:44249 every 8 connection(s)
I20260809 08:56:01.586436 4164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:56:01.587425 4275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:01.587757 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 4164
I20260809 08:56:01.587847 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:56:01.588133 8808 external_mini_cluster.cc:1428] Setting key 4199c29b872d438606fb0b6c93e92a3f
I20260809 08:56:01.590165 4275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53e9822cd6054a2bbf0c5e5409e1fe2f: Bootstrap starting.
I20260809 08:56:01.590787 4275 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 53e9822cd6054a2bbf0c5e5409e1fe2f: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:01.591087 4275 log.cc:824] T 00000000000000000000000000000000 P 53e9822cd6054a2bbf0c5e5409e1fe2f: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:01.591804 4275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 53e9822cd6054a2bbf0c5e5409e1fe2f: No bootstrap required, opened a new log
I20260809 08:56:01.593820 4279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:01.588719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55664 (local address 127.8.154.62:44249)
0809 08:56:01.588938 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:01.588945 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:01.588971 (+ 26us) server_negotiation.cc:407] Connection header received
0809 08:56:01.589403 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:01.589409 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:01.589470 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:01.589629 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:01.589848 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.590773 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.591767 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.592289 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.592421 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:01.593214 (+ 793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:01.593218 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:01.593218 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:56:01.593381 (+ 163us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:01.593413 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:56:01.593424 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:01.593494 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:01.593571 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 08:56:01.593652 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":88,"threads_started":1}
I20260809 08:56:01.594025 4275 raft_consensus.cc:348] T 00000000000000000000000000000000 P 53e9822cd6054a2bbf0c5e5409e1fe2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53e9822cd6054a2bbf0c5e5409e1fe2f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 44249 } }
I20260809 08:56:01.594213 4275 raft_consensus.cc:374] T 00000000000000000000000000000000 P 53e9822cd6054a2bbf0c5e5409e1fe2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:01.594265 4275 raft_consensus.cc:729] T 00000000000000000000000000000000 P 53e9822cd6054a2bbf0c5e5409e1fe2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53e9822cd6054a2bbf0c5e5409e1fe2f, State: Initialized, Role: FOLLOWER
I20260809 08:56:01.594393 4275 consensus_queue.cc:260] T 00000000000000000000000000000000 P 53e9822cd6054a2bbf0c5e5409e1fe2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53e9822cd6054a2bbf0c5e5409e1fe2f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 44249 } }
I20260809 08:56:01.594512 4275 raft_consensus.cc:388] T 00000000000000000000000000000000 P 53e9822cd6054a2bbf0c5e5409e1fe2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:56:01.594568 4275 raft_consensus.cc:482] T 00000000000000000000000000000000 P 53e9822cd6054a2bbf0c5e5409e1fe2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:56:01.594624 4275 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 53e9822cd6054a2bbf0c5e5409e1fe2f [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:01.595162 4275 raft_consensus.cc:504] T 00000000000000000000000000000000 P 53e9822cd6054a2bbf0c5e5409e1fe2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53e9822cd6054a2bbf0c5e5409e1fe2f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 44249 } }
I20260809 08:56:01.595273 4275 leader_election.cc:302] T 00000000000000000000000000000000 P 53e9822cd6054a2bbf0c5e5409e1fe2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 53e9822cd6054a2bbf0c5e5409e1fe2f; no voters:
I20260809 08:56:01.595444 4275 leader_election.cc:288] T 00000000000000000000000000000000 P 53e9822cd6054a2bbf0c5e5409e1fe2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:56:01.595492 4281 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 53e9822cd6054a2bbf0c5e5409e1fe2f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:56:01.595669 4281 raft_consensus.cc:686] T 00000000000000000000000000000000 P 53e9822cd6054a2bbf0c5e5409e1fe2f [term 1 LEADER]: Becoming Leader. State: Replica: 53e9822cd6054a2bbf0c5e5409e1fe2f, State: Running, Role: LEADER
I20260809 08:56:01.595739 4275 sys_catalog.cc:564] T 00000000000000000000000000000000 P 53e9822cd6054a2bbf0c5e5409e1fe2f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:56:01.595801 4281 consensus_queue.cc:237] T 00000000000000000000000000000000 P 53e9822cd6054a2bbf0c5e5409e1fe2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53e9822cd6054a2bbf0c5e5409e1fe2f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 44249 } }
I20260809 08:56:01.596165 4283 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53e9822cd6054a2bbf0c5e5409e1fe2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "53e9822cd6054a2bbf0c5e5409e1fe2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53e9822cd6054a2bbf0c5e5409e1fe2f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 44249 } } }
I20260809 08:56:01.596174 4284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 53e9822cd6054a2bbf0c5e5409e1fe2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 53e9822cd6054a2bbf0c5e5409e1fe2f. Latest consensus state: current_term: 1 leader_uuid: "53e9822cd6054a2bbf0c5e5409e1fe2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "53e9822cd6054a2bbf0c5e5409e1fe2f" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 44249 } } }
I20260809 08:56:01.596268 4283 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53e9822cd6054a2bbf0c5e5409e1fe2f [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:01.596302 4284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 53e9822cd6054a2bbf0c5e5409e1fe2f [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:01.596493 4292 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:56:01.597004 4292 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:56:01.598542 4292 catalog_manager.cc:1347] Generated new cluster ID: 04ac246dacee4f5583b028908cd94329
I20260809 08:56:01.598601 4292 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:56:01.611246 4292 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:56:01.611960 4292 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:56:01.617669 4292 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 53e9822cd6054a2bbf0c5e5409e1fe2f: Generated new TSK 0
I20260809 08:56:01.617847 4292 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:56:01.620049 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.1:0
--local_ip_for_outbound_sockets=127.8.154.1
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:44249
--builtin_ntp_servers=127.8.154.20:36389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 08:56:01.709913 4305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:01.710155 4305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:01.710196 4305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:01.711369 4305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:01.711436 4305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:01.711481 4305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.1
I20260809 08:56:01.712939 4305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:36389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:44249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.1
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:01.713224 4305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:01.713549 4305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:01.719584 4316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:01.719641 4312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:01.719681 4321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:01.720052 4305 server_base.cc:1034] running on GCE node
I20260809 08:56:01.720292 4305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:01.720547 4305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:01.721745 4305 hybrid_clock.cc:648] HybridClock initialized: now 1786265761721749 us; error 63 us; skew 500 ppm
I20260809 08:56:01.723261 4305 webserver.cc:460] Webserver started at http://127.8.154.1:41351/ using document root <none> and password file <none>
I20260809 08:56:01.723563 4305 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:01.723642 4305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:01.723817 4305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:01.725262 4305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-0/data/instance:
uuid: "f2806c613ed14dfebb8f027fdedf25b1"
format_stamp: "Formatted at 2026-08-09 08:56:01 on dist-test-slave-t3q3"
server_key: "1ab0f69f068bb2bb74c226bbc24cbec0"
server_key_iv: "c82e198908dba655bf20038494b154f2"
server_key_version: "encryptionkey@0"
I20260809 08:56:01.725689 4305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance:
uuid: "f2806c613ed14dfebb8f027fdedf25b1"
format_stamp: "Formatted at 2026-08-09 08:56:01 on dist-test-slave-t3q3"
server_key: "1ab0f69f068bb2bb74c226bbc24cbec0"
server_key_iv: "c82e198908dba655bf20038494b154f2"
server_key_version: "encryptionkey@0"
I20260809 08:56:01.727128 4305 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:56:01.727995 4329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:01.728235 4305 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:56:01.728355 4305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-0/wal
uuid: "f2806c613ed14dfebb8f027fdedf25b1"
format_stamp: "Formatted at 2026-08-09 08:56:01 on dist-test-slave-t3q3"
server_key: "1ab0f69f068bb2bb74c226bbc24cbec0"
server_key_iv: "c82e198908dba655bf20038494b154f2"
server_key_version: "encryptionkey@0"
I20260809 08:56:01.728443 4305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:01.746609 4305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:01.746942 4305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:01.747095 4305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:01.747332 4305 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:56:01.747675 4305 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:56:01.747728 4305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:01.747771 4305 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:56:01.747797 4305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:01.753767 4305 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.1:38643
I20260809 08:56:01.753847 4502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.1:38643 every 8 connection(s)
I20260809 08:56:01.754176 4305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
I20260809 08:56:01.755926 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 4305
I20260809 08:56:01.756094 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance
I20260809 08:56:01.756306 8808 external_mini_cluster.cc:1428] Setting key 309adcb52ca198915ee80c91e86694ea
I20260809 08:56:01.757254 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.2:0
--local_ip_for_outbound_sockets=127.8.154.2
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:44249
--builtin_ntp_servers=127.8.154.20:36389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 08:56:01.759775 4279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:01.754869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:55185 (local address 127.8.154.62:44249)
0809 08:56:01.754933 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:01.754935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:01.755136 (+ 201us) server_negotiation.cc:407] Connection header received
0809 08:56:01.755693 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:01.755694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:01.755712 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:01.755774 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:01.756624 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.757149 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.758254 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.758456 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.758534 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:01.759486 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:01.759487 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:01.759488 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:56:01.759493 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:01.759512 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:56:01.759517 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:01.759582 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:01.759691 (+ 109us) server_negotiation.cc:299] Negotiation successful
0809 08:56:01.759715 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 08:56:01.760053 4505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:01.754834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:44249 (local address 127.8.154.1:55185)
0809 08:56:01.755009 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:01.755043 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:01.755570 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:01.755862 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:01.755866 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:01.755873 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:56:01.756526 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:01.756533 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.757249 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.757250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:01.758160 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:01.758162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.758542 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.758544 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:01.758720 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:01.759309 (+ 589us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:56:01.759310 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:56:01.759351 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:56:01.759382 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:56:01.759593 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:56:01.759598 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:56:01.759602 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:56:01.759682 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 08:56:01.759797 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":67,"threads_started":1}
I20260809 08:56:01.760639 4503 heartbeater.cc:347] Connected to a master server at 127.8.154.62:44249
I20260809 08:56:01.760740 4503 heartbeater.cc:464] Registering TS with master...
I20260809 08:56:01.760942 4503 heartbeater.cc:511] Master 127.8.154.62:44249 requested a full tablet report, sending...
I20260809 08:56:01.761432 4223 ts_manager.cc:194] Registered new tserver with Master: f2806c613ed14dfebb8f027fdedf25b1 (127.8.154.1:38643)
I20260809 08:56:01.762142 4223 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.154.1:55185
W20260809 08:56:01.841161 4506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:01.841393 4506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:01.841431 4506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:01.842563 4506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:01.842635 4506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:01.842679 4506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.2
I20260809 08:56:01.844035 4506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:36389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:44249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.2
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:01.844344 4506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:01.844648 4506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:01.849591 4516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:01.849560 4518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:01.849565 4515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:01.849833 4506 server_base.cc:1034] running on GCE node
I20260809 08:56:01.850095 4506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:01.850337 4506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:01.851487 4506 hybrid_clock.cc:648] HybridClock initialized: now 1786265761851475 us; error 29 us; skew 500 ppm
I20260809 08:56:01.852789 4506 webserver.cc:460] Webserver started at http://127.8.154.2:35081/ using document root <none> and password file <none>
I20260809 08:56:01.853063 4506 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:01.853125 4506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:01.853268 4506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:01.854375 4506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-1/data/instance:
uuid: "aa01e66b3b06413ca1b2f35391b70c78"
format_stamp: "Formatted at 2026-08-09 08:56:01 on dist-test-slave-t3q3"
server_key: "bcd9adda57709975fdce720623c01a77"
server_key_iv: "a0e4800dfdcbd84fd7f6818cdab6c2b5"
server_key_version: "encryptionkey@0"
I20260809 08:56:01.854733 4506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance:
uuid: "aa01e66b3b06413ca1b2f35391b70c78"
format_stamp: "Formatted at 2026-08-09 08:56:01 on dist-test-slave-t3q3"
server_key: "bcd9adda57709975fdce720623c01a77"
server_key_iv: "a0e4800dfdcbd84fd7f6818cdab6c2b5"
server_key_version: "encryptionkey@0"
I20260809 08:56:01.856029 4506 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:56:01.856770 4525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:01.856930 4506 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:56:01.857031 4506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-1/wal
uuid: "aa01e66b3b06413ca1b2f35391b70c78"
format_stamp: "Formatted at 2026-08-09 08:56:01 on dist-test-slave-t3q3"
server_key: "bcd9adda57709975fdce720623c01a77"
server_key_iv: "a0e4800dfdcbd84fd7f6818cdab6c2b5"
server_key_version: "encryptionkey@0"
I20260809 08:56:01.857115 4506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:01.867493 4506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:01.867779 4506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:01.867918 4506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:01.868203 4506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:56:01.868557 4506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:56:01.868616 4506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:01.868659 4506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:56:01.868685 4506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:01.874621 4506 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.2:34953
I20260809 08:56:01.874665 4696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.2:34953 every 8 connection(s)
I20260809 08:56:01.875087 4506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
I20260809 08:56:01.880174 4279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:01.875852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:54871 (local address 127.8.154.62:44249)
0809 08:56:01.875942 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:01.875944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:01.876132 (+ 188us) server_negotiation.cc:407] Connection header received
0809 08:56:01.876629 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:01.876630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:01.876641 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:01.876704 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:01.877360 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.877873 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.878835 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.879045 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.879098 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:01.879856 (+ 758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:01.879858 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:01.879858 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:56:01.879862 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:01.879879 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:56:01.879891 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:01.879945 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:01.880021 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 08:56:01.880043 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 08:56:01.880409 4700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:01.875804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:44249 (local address 127.8.154.2:54871)
0809 08:56:01.875993 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:01.876035 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:01.876543 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:01.876758 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:01.876762 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:01.876769 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:56:01.877281 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:01.877286 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.877961 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.877962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:01.878763 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:01.878765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:01.879130 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:01.879131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:01.879243 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:01.879722 (+ 479us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:56:01.879723 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:56:01.879750 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:56:01.879772 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:56:01.879953 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:56:01.879957 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:56:01.879960 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:56:01.880011 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 08:56:01.880166 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":90,"threads_started":1}
I20260809 08:56:01.880904 4697 heartbeater.cc:347] Connected to a master server at 127.8.154.62:44249
I20260809 08:56:01.881011 4697 heartbeater.cc:464] Registering TS with master...
I20260809 08:56:01.881210 4697 heartbeater.cc:511] Master 127.8.154.62:44249 requested a full tablet report, sending...
I20260809 08:56:01.881598 4223 ts_manager.cc:194] Registered new tserver with Master: aa01e66b3b06413ca1b2f35391b70c78 (127.8.154.2:34953)
I20260809 08:56:01.881929 4223 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.154.2:54871
I20260809 08:56:01.883550 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 4506
I20260809 08:56:01.883651 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance
I20260809 08:56:01.883831 8808 external_mini_cluster.cc:1428] Setting key 96f387f07d5ab35fd7e4582c09ea305d
I20260809 08:56:01.884708 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.3:0
--local_ip_for_outbound_sockets=127.8.154.3
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:44249
--builtin_ntp_servers=127.8.154.20:36389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 08:56:01.967672 4701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:01.967871 4701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:01.967892 4701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:01.969065 4701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:01.969128 4701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:01.969152 4701 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.3
I20260809 08:56:01.970423 4701 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:36389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:44249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.3
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:01.970615 4701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:01.970862 4701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:01.975988 4713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:01.976027 4711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:01.976243 4701 server_base.cc:1034] running on GCE node
W20260809 08:56:01.976001 4715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:01.976547 4701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:01.976750 4701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:01.977885 4701 hybrid_clock.cc:648] HybridClock initialized: now 1786265761977875 us; error 28 us; skew 500 ppm
I20260809 08:56:01.979131 4701 webserver.cc:460] Webserver started at http://127.8.154.3:36199/ using document root <none> and password file <none>
I20260809 08:56:01.979346 4701 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:01.979388 4701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:01.979477 4701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:01.980592 4701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-2/data/instance:
uuid: "63dd09310b3b4d9690ada36e8add1807"
format_stamp: "Formatted at 2026-08-09 08:56:01 on dist-test-slave-t3q3"
server_key: "3bee478b8a888781e7d50ffc8a71b87b"
server_key_iv: "54fda2ef6c1317a810e503363bd319ca"
server_key_version: "encryptionkey@0"
I20260809 08:56:01.980893 4701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance:
uuid: "63dd09310b3b4d9690ada36e8add1807"
format_stamp: "Formatted at 2026-08-09 08:56:01 on dist-test-slave-t3q3"
server_key: "3bee478b8a888781e7d50ffc8a71b87b"
server_key_iv: "54fda2ef6c1317a810e503363bd319ca"
server_key_version: "encryptionkey@0"
I20260809 08:56:01.982183 4701 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:56:01.982856 4722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:01.983063 4701 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:56:01.983152 4701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-2/wal
uuid: "63dd09310b3b4d9690ada36e8add1807"
format_stamp: "Formatted at 2026-08-09 08:56:01 on dist-test-slave-t3q3"
server_key: "3bee478b8a888781e7d50ffc8a71b87b"
server_key_iv: "54fda2ef6c1317a810e503363bd319ca"
server_key_version: "encryptionkey@0"
I20260809 08:56:01.983227 4701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:01.995775 4701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:01.996167 4701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:01.996428 4701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:01.996657 4701 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:56:01.997027 4701 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:56:01.997092 4701 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:01.997123 4701 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:56:01.997149 4701 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:02.002734 4701 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.3:35601
I20260809 08:56:02.002827 4895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.3:35601 every 8 connection(s)
I20260809 08:56:02.003144 4701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
I20260809 08:56:02.008279 4279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:02.003905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:58137 (local address 127.8.154.62:44249)
0809 08:56:02.003951 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:02.003953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:02.004222 (+ 269us) server_negotiation.cc:407] Connection header received
0809 08:56:02.004749 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:02.004750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:02.004759 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:02.004819 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:02.005484 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:02.005987 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:02.006980 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:02.007169 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:02.007225 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:02.007984 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:02.007985 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:02.007986 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:56:02.007990 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:02.008003 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:56:02.008006 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:02.008051 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:02.008197 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 08:56:02.008219 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:56:02.008512 4898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:02.003870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:44249 (local address 127.8.154.3:58137)
0809 08:56:02.004094 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:02.004135 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:02.004650 (+ 515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:02.004879 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:02.004882 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:02.004890 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:56:02.005412 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:02.005416 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:02.006091 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:02.006092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:02.006902 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:02.006905 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:02.007248 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:02.007249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:02.007362 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:02.007856 (+ 494us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:56:02.007856 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:56:02.007884 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:56:02.007907 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:56:02.008108 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:56:02.008113 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:56:02.008116 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:56:02.008184 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 08:56:02.008268 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":90,"threads_started":1}
I20260809 08:56:02.009132 4896 heartbeater.cc:347] Connected to a master server at 127.8.154.62:44249
I20260809 08:56:02.009238 4896 heartbeater.cc:464] Registering TS with master...
I20260809 08:56:02.009418 4896 heartbeater.cc:511] Master 127.8.154.62:44249 requested a full tablet report, sending...
I20260809 08:56:02.009814 4223 ts_manager.cc:194] Registered new tserver with Master: 63dd09310b3b4d9690ada36e8add1807 (127.8.154.3:35601)
I20260809 08:56:02.010193 4223 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.154.3:58137
I20260809 08:56:02.010403 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 4701
I20260809 08:56:02.010524 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance
I20260809 08:56:02.010740 8808 external_mini_cluster.cc:1428] Setting key 11c46da1a0a2adabcdff25d6a05b9251
I20260809 08:56:02.011682 8808 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:56:02.015854 4279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:02.013065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55680 (local address 127.8.154.62:44249)
0809 08:56:02.013143 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:02.013145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:02.013240 (+ 95us) server_negotiation.cc:407] Connection header received
0809 08:56:02.013316 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:02.013317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:02.013327 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:02.013383 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:02.013541 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:02.014005 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:02.014742 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:02.014943 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:02.015015 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:02.015612 (+ 597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:02.015613 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:02.015614 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:56:02.015618 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:02.015631 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:56:02.015635 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:02.015677 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:02.015751 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 08:56:02.015771 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 08:56:02.017231 4223 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55680:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:56:02.018725 4223 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:56:02.025221 4913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:02.021142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:38643 (local address 127.0.0.1:42116)
0809 08:56:02.021649 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:02.021659 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:02.021718 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:02.021963 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:02.021964 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:02.021966 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:56:02.022014 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:02.022015 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:02.022886 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:02.022886 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:02.024016 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:02.024018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:02.024904 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:02.024906 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:02.024987 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:02.025007 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 08:56:02.025080 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 08:56:02.025140 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"spinlock_wait_cycles":768,"thread_start_us":396,"threads_started":1}
I20260809 08:56:02.025233 4908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:02.021322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42116 (local address 127.8.154.1:38643)
0809 08:56:02.021518 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:02.021523 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:02.021715 (+ 192us) server_negotiation.cc:407] Connection header received
0809 08:56:02.021786 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:02.021788 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:02.021820 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:02.021891 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:56:02.022101 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:02.022804 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:02.024146 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:02.024815 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:02.024893 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:02.024982 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:02.025105 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 08:56:02.025145 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":96,"threads_started":1}
I20260809 08:56:02.025568 4911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:02.021346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36358 (local address 127.8.154.2:34953)
0809 08:56:02.021696 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:02.021700 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:02.021709 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:56:02.021748 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:02.021750 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:02.021765 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:02.021845 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:56:02.022035 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:02.023163 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:02.024291 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:02.024949 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:02.025016 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:02.025104 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:02.025449 (+ 345us) server_negotiation.cc:299] Negotiation successful
0809 08:56:02.025490 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":90,"threads_started":1}
I20260809 08:56:02.025388 4912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:02.021136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:34953 (local address 127.0.0.1:36358)
0809 08:56:02.021570 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:02.021579 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:02.021634 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:02.021917 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:02.021917 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:02.021919 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:56:02.021957 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:02.021958 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:02.023279 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:02.023280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:02.024204 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:02.024207 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:02.025139 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:02.025140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:02.025194 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:02.025209 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 08:56:02.025262 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 08:56:02.025302 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":463,"threads_started":1}
I20260809 08:56:02.025765 4905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:02.021095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:35601 (local address 127.0.0.1:34852)
0809 08:56:02.021295 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:02.021334 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:02.021396 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:02.021795 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:02.021799 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:02.021802 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:56:02.021887 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:02.021890 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:02.023262 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:02.023263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:02.024528 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:02.024531 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:02.025504 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:02.025506 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:02.025568 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:02.025572 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:56:02.025633 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 08:56:02.025683 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":12,"thread_start_us":81,"threads_started":1}
I20260809 08:56:02.026049 4907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:02.021180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34852 (local address 127.8.154.3:35601)
0809 08:56:02.021561 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:02.021566 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:02.021579 (+ 13us) server_negotiation.cc:407] Connection header received
0809 08:56:02.021642 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:02.021644 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:02.021661 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:02.021971 (+ 310us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:56:02.022029 (+ 58us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:02.023165 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:02.024622 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:02.025403 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:02.025490 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:02.025571 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:02.025934 (+ 363us) server_negotiation.cc:299] Negotiation successful
0809 08:56:02.025975 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":104,"threads_started":1}
I20260809 08:56:02.026335 4403 tablet_service.cc:1467] Processing CreateTablet for tablet 8b0c4e33ee274476a9e85c0b3110f7bc (DEFAULT_TABLE table=test-table [id=dfb81ae50fc54c389abcdcb0e39dffb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:56:02.026544 4604 tablet_service.cc:1467] Processing CreateTablet for tablet 8b0c4e33ee274476a9e85c0b3110f7bc (DEFAULT_TABLE table=test-table [id=dfb81ae50fc54c389abcdcb0e39dffb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:56:02.026736 4403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b0c4e33ee274476a9e85c0b3110f7bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:02.026899 4604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b0c4e33ee274476a9e85c0b3110f7bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:02.026849 4796 tablet_service.cc:1467] Processing CreateTablet for tablet 8b0c4e33ee274476a9e85c0b3110f7bc (DEFAULT_TABLE table=test-table [id=dfb81ae50fc54c389abcdcb0e39dffb1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:56:02.027177 4796 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b0c4e33ee274476a9e85c0b3110f7bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:02.029501 4914 tablet_bootstrap.cc:492] T 8b0c4e33ee274476a9e85c0b3110f7bc P f2806c613ed14dfebb8f027fdedf25b1: Bootstrap starting.
I20260809 08:56:02.030256 4914 tablet_bootstrap.cc:654] T 8b0c4e33ee274476a9e85c0b3110f7bc P f2806c613ed14dfebb8f027fdedf25b1: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:02.030536 4917 tablet_bootstrap.cc:492] T 8b0c4e33ee274476a9e85c0b3110f7bc P 63dd09310b3b4d9690ada36e8add1807: Bootstrap starting.
I20260809 08:56:02.030697 4916 tablet_bootstrap.cc:492] T 8b0c4e33ee274476a9e85c0b3110f7bc P aa01e66b3b06413ca1b2f35391b70c78: Bootstrap starting.
I20260809 08:56:02.030869 4914 log.cc:824] T 8b0c4e33ee274476a9e85c0b3110f7bc P f2806c613ed14dfebb8f027fdedf25b1: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:02.031473 4916 tablet_bootstrap.cc:654] T 8b0c4e33ee274476a9e85c0b3110f7bc P aa01e66b3b06413ca1b2f35391b70c78: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:02.031585 4917 tablet_bootstrap.cc:654] T 8b0c4e33ee274476a9e85c0b3110f7bc P 63dd09310b3b4d9690ada36e8add1807: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:02.031951 4916 log.cc:824] T 8b0c4e33ee274476a9e85c0b3110f7bc P aa01e66b3b06413ca1b2f35391b70c78: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:02.032018 4917 log.cc:824] T 8b0c4e33ee274476a9e85c0b3110f7bc P 63dd09310b3b4d9690ada36e8add1807: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:02.032743 4916 tablet_bootstrap.cc:492] T 8b0c4e33ee274476a9e85c0b3110f7bc P aa01e66b3b06413ca1b2f35391b70c78: No bootstrap required, opened a new log
I20260809 08:56:02.032815 4916 ts_tablet_manager.cc:1397] T 8b0c4e33ee274476a9e85c0b3110f7bc P aa01e66b3b06413ca1b2f35391b70c78: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:56:02.032923 4914 tablet_bootstrap.cc:492] T 8b0c4e33ee274476a9e85c0b3110f7bc P f2806c613ed14dfebb8f027fdedf25b1: No bootstrap required, opened a new log
I20260809 08:56:02.033005 4914 ts_tablet_manager.cc:1397] T 8b0c4e33ee274476a9e85c0b3110f7bc P f2806c613ed14dfebb8f027fdedf25b1: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 08:56:02.034722 4916 raft_consensus.cc:348] T 8b0c4e33ee274476a9e85c0b3110f7bc P aa01e66b3b06413ca1b2f35391b70c78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63dd09310b3b4d9690ada36e8add1807" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 35601 } } peers { permanent_uuid: "f2806c613ed14dfebb8f027fdedf25b1" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 38643 } } peers { permanent_uuid: "aa01e66b3b06413ca1b2f35391b70c78" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 34953 } }
I20260809 08:56:02.034853 4916 raft_consensus.cc:374] T 8b0c4e33ee274476a9e85c0b3110f7bc P aa01e66b3b06413ca1b2f35391b70c78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:02.034876 4916 raft_consensus.cc:729] T 8b0c4e33ee274476a9e85c0b3110f7bc P aa01e66b3b06413ca1b2f35391b70c78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa01e66b3b06413ca1b2f35391b70c78, State: Initialized, Role: FOLLOWER
I20260809 08:56:02.034988 4916 consensus_queue.cc:260] T 8b0c4e33ee274476a9e85c0b3110f7bc P aa01e66b3b06413ca1b2f35391b70c78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63dd09310b3b4d9690ada36e8add1807" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 35601 } } peers { permanent_uuid: "f2806c613ed14dfebb8f027fdedf25b1" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 38643 } } peers { permanent_uuid: "aa01e66b3b06413ca1b2f35391b70c78" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 34953 } }
I20260809 08:56:02.035048 4914 raft_consensus.cc:348] T 8b0c4e33ee274476a9e85c0b3110f7bc P f2806c613ed14dfebb8f027fdedf25b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63dd09310b3b4d9690ada36e8add1807" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 35601 } } peers { permanent_uuid: "f2806c613ed14dfebb8f027fdedf25b1" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 38643 } } peers { permanent_uuid: "aa01e66b3b06413ca1b2f35391b70c78" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 34953 } }
I20260809 08:56:02.035236 4914 raft_consensus.cc:374] T 8b0c4e33ee274476a9e85c0b3110f7bc P f2806c613ed14dfebb8f027fdedf25b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:02.035257 4916 ts_tablet_manager.cc:1428] T 8b0c4e33ee274476a9e85c0b3110f7bc P aa01e66b3b06413ca1b2f35391b70c78: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 08:56:02.035281 4697 heartbeater.cc:503] Master 127.8.154.62:44249 was elected leader, sending a full tablet report...
I20260809 08:56:02.035286 4914 raft_consensus.cc:729] T 8b0c4e33ee274476a9e85c0b3110f7bc P f2806c613ed14dfebb8f027fdedf25b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2806c613ed14dfebb8f027fdedf25b1, State: Initialized, Role: FOLLOWER
I20260809 08:56:02.035403 4914 consensus_queue.cc:260] T 8b0c4e33ee274476a9e85c0b3110f7bc P f2806c613ed14dfebb8f027fdedf25b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63dd09310b3b4d9690ada36e8add1807" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 35601 } } peers { permanent_uuid: "f2806c613ed14dfebb8f027fdedf25b1" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 38643 } } peers { permanent_uuid: "aa01e66b3b06413ca1b2f35391b70c78" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 34953 } }
I20260809 08:56:02.035668 4914 ts_tablet_manager.cc:1428] T 8b0c4e33ee274476a9e85c0b3110f7bc P f2806c613ed14dfebb8f027fdedf25b1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:56:02.035746 4917 tablet_bootstrap.cc:492] T 8b0c4e33ee274476a9e85c0b3110f7bc P 63dd09310b3b4d9690ada36e8add1807: No bootstrap required, opened a new log
I20260809 08:56:02.035825 4917 ts_tablet_manager.cc:1397] T 8b0c4e33ee274476a9e85c0b3110f7bc P 63dd09310b3b4d9690ada36e8add1807: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260809 08:56:02.035817 4503 heartbeater.cc:503] Master 127.8.154.62:44249 was elected leader, sending a full tablet report...
I20260809 08:56:02.037832 4917 raft_consensus.cc:348] T 8b0c4e33ee274476a9e85c0b3110f7bc P 63dd09310b3b4d9690ada36e8add1807 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63dd09310b3b4d9690ada36e8add1807" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 35601 } } peers { permanent_uuid: "f2806c613ed14dfebb8f027fdedf25b1" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 38643 } } peers { permanent_uuid: "aa01e66b3b06413ca1b2f35391b70c78" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 34953 } }
I20260809 08:56:02.037955 4917 raft_consensus.cc:374] T 8b0c4e33ee274476a9e85c0b3110f7bc P 63dd09310b3b4d9690ada36e8add1807 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:02.037977 4917 raft_consensus.cc:729] T 8b0c4e33ee274476a9e85c0b3110f7bc P 63dd09310b3b4d9690ada36e8add1807 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63dd09310b3b4d9690ada36e8add1807, State: Initialized, Role: FOLLOWER
I20260809 08:56:02.038080 4917 consensus_queue.cc:260] T 8b0c4e33ee274476a9e85c0b3110f7bc P 63dd09310b3b4d9690ada36e8add1807 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63dd09310b3b4d9690ada36e8add1807" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 35601 } } peers { permanent_uuid: "f2806c613ed14dfebb8f027fdedf25b1" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 38643 } } peers { permanent_uuid: "aa01e66b3b06413ca1b2f35391b70c78" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 34953 } }
I20260809 08:56:02.038340 4917 ts_tablet_manager.cc:1428] T 8b0c4e33ee274476a9e85c0b3110f7bc P 63dd09310b3b4d9690ada36e8add1807: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 08:56:02.038404 4896 heartbeater.cc:503] Master 127.8.154.62:44249 was elected leader, sending a full tablet report...
W20260809 08:56:02.125680 4698 tablet.cc:2369] T 8b0c4e33ee274476a9e85c0b3110f7bc P aa01e66b3b06413ca1b2f35391b70c78: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:56:02.243245 4923 raft_consensus.cc:482] T 8b0c4e33ee274476a9e85c0b3110f7bc P f2806c613ed14dfebb8f027fdedf25b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:56:02.243427 4923 raft_consensus.cc:504] T 8b0c4e33ee274476a9e85c0b3110f7bc P f2806c613ed14dfebb8f027fdedf25b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63dd09310b3b4d9690ada36e8add1807" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 35601 } } peers { permanent_uuid: "f2806c613ed14dfebb8f027fdedf25b1" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 38643 } } peers { permanent_uuid: "aa01e66b3b06413ca1b2f35391b70c78" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 34953 } }
I20260809 08:56:02.243848 4923 leader_election.cc:288] T 8b0c4e33ee274476a9e85c0b3110f7bc P f2806c613ed14dfebb8f027fdedf25b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 63dd09310b3b4d9690ada36e8add1807 (127.8.154.3:35601), aa01e66b3b06413ca1b2f35391b70c78 (127.8.154.2:34953)
I20260809 08:56:02.246661 4505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:02.244048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:35601 (local address 127.8.154.1:40527)
0809 08:56:02.244144 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:02.244159 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:02.244214 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:02.244359 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:02.244359 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:02.244361 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:56:02.244442 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:02.244444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:02.245098 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:02.245099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:02.245854 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:02.245857 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:02.246408 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:02.246409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:02.246479 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:02.246482 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:56:02.246546 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 08:56:02.246586 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 08:56:02.246658 4907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:02.244173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:40527 (local address 127.8.154.3:35601)
0809 08:56:02.244221 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:02.244223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:02.244249 (+ 26us) server_negotiation.cc:407] Connection header received
0809 08:56:02.244281 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:02.244281 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:02.244296 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:02.244354 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:56:02.244517 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:02.245012 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:02.245950 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:02.246314 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:02.246407 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:02.246418 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:02.246552 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 08:56:02.246589 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 08:56:02.246919 4911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:02.244183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:37917 (local address 127.8.154.2:34953)
0809 08:56:02.244234 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:02.244237 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:02.244338 (+ 101us) server_negotiation.cc:407] Connection header received
0809 08:56:02.244413 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:02.244414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:02.244424 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:02.244491 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:56:02.244649 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:02.245117 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:02.246032 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:02.246514 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:02.246596 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:02.246607 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:02.246800 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 08:56:02.246856 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 08:56:02.247051 4828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b0c4e33ee274476a9e85c0b3110f7bc" candidate_uuid: "f2806c613ed14dfebb8f027fdedf25b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63dd09310b3b4d9690ada36e8add1807" is_pre_election: true
I20260809 08:56:02.247133 4925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:02.244091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:34953 (local address 127.8.154.1:37917)
0809 08:56:02.244264 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:02.244273 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:02.244353 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:02.244497 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:02.244498 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:02.244500 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:56:02.244577 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:02.244578 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:02.245200 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:02.245201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:02.245941 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:02.245942 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:02.246610 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:02.246611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:02.246688 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:02.246691 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:56:02.247000 (+ 309us) client_negotiation.cc:240] Negotiation successful
0809 08:56:02.247043 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":98,"threads_started":1}
I20260809 08:56:02.247238 4828 raft_consensus.cc:2454] T 8b0c4e33ee274476a9e85c0b3110f7bc P 63dd09310b3b4d9690ada36e8add1807 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2806c613ed14dfebb8f027fdedf25b1 in term 0.
I20260809 08:56:02.247498 4336 leader_election.cc:302] T 8b0c4e33ee274476a9e85c0b3110f7bc P f2806c613ed14dfebb8f027fdedf25b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63dd09310b3b4d9690ada36e8add1807, f2806c613ed14dfebb8f027fdedf25b1; no voters:
I20260809 08:56:02.247543 4633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b0c4e33ee274476a9e85c0b3110f7bc" candidate_uuid: "f2806c613ed14dfebb8f027fdedf25b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa01e66b3b06413ca1b2f35391b70c78" is_pre_election: true
I20260809 08:56:02.247710 4633 raft_consensus.cc:2454] T 8b0c4e33ee274476a9e85c0b3110f7bc P aa01e66b3b06413ca1b2f35391b70c78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2806c613ed14dfebb8f027fdedf25b1 in term 0.
I20260809 08:56:02.247773 4923 raft_consensus.cc:2781] T 8b0c4e33ee274476a9e85c0b3110f7bc P f2806c613ed14dfebb8f027fdedf25b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:56:02.247876 4923 raft_consensus.cc:482] T 8b0c4e33ee274476a9e85c0b3110f7bc P f2806c613ed14dfebb8f027fdedf25b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:56:02.247913 4923 raft_consensus.cc:3037] T 8b0c4e33ee274476a9e85c0b3110f7bc P f2806c613ed14dfebb8f027fdedf25b1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:02.248718 4923 raft_consensus.cc:504] T 8b0c4e33ee274476a9e85c0b3110f7bc P f2806c613ed14dfebb8f027fdedf25b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63dd09310b3b4d9690ada36e8add1807" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 35601 } } peers { permanent_uuid: "f2806c613ed14dfebb8f027fdedf25b1" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 38643 } } peers { permanent_uuid: "aa01e66b3b06413ca1b2f35391b70c78" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 34953 } }
I20260809 08:56:02.248872 4923 leader_election.cc:288] T 8b0c4e33ee274476a9e85c0b3110f7bc P f2806c613ed14dfebb8f027fdedf25b1 [CANDIDATE]: Term 1 election: Requested vote from peers 63dd09310b3b4d9690ada36e8add1807 (127.8.154.3:35601), aa01e66b3b06413ca1b2f35391b70c78 (127.8.154.2:34953)
I20260809 08:56:02.249040 4633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b0c4e33ee274476a9e85c0b3110f7bc" candidate_uuid: "f2806c613ed14dfebb8f027fdedf25b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa01e66b3b06413ca1b2f35391b70c78"
I20260809 08:56:02.249042 4828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b0c4e33ee274476a9e85c0b3110f7bc" candidate_uuid: "f2806c613ed14dfebb8f027fdedf25b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63dd09310b3b4d9690ada36e8add1807"
I20260809 08:56:02.249121 4828 raft_consensus.cc:3037] T 8b0c4e33ee274476a9e85c0b3110f7bc P 63dd09310b3b4d9690ada36e8add1807 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:02.249121 4633 raft_consensus.cc:3037] T 8b0c4e33ee274476a9e85c0b3110f7bc P aa01e66b3b06413ca1b2f35391b70c78 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:02.250052 4633 raft_consensus.cc:2454] T 8b0c4e33ee274476a9e85c0b3110f7bc P aa01e66b3b06413ca1b2f35391b70c78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2806c613ed14dfebb8f027fdedf25b1 in term 1.
I20260809 08:56:02.250053 4828 raft_consensus.cc:2454] T 8b0c4e33ee274476a9e85c0b3110f7bc P 63dd09310b3b4d9690ada36e8add1807 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2806c613ed14dfebb8f027fdedf25b1 in term 1.
I20260809 08:56:02.250231 4336 leader_election.cc:302] T 8b0c4e33ee274476a9e85c0b3110f7bc P f2806c613ed14dfebb8f027fdedf25b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 63dd09310b3b4d9690ada36e8add1807, f2806c613ed14dfebb8f027fdedf25b1; no voters:
I20260809 08:56:02.250334 4923 raft_consensus.cc:2781] T 8b0c4e33ee274476a9e85c0b3110f7bc P f2806c613ed14dfebb8f027fdedf25b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:56:02.250476 4923 raft_consensus.cc:686] T 8b0c4e33ee274476a9e85c0b3110f7bc P f2806c613ed14dfebb8f027fdedf25b1 [term 1 LEADER]: Becoming Leader. State: Replica: f2806c613ed14dfebb8f027fdedf25b1, State: Running, Role: LEADER
I20260809 08:56:02.250589 4923 consensus_queue.cc:237] T 8b0c4e33ee274476a9e85c0b3110f7bc P f2806c613ed14dfebb8f027fdedf25b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63dd09310b3b4d9690ada36e8add1807" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 35601 } } peers { permanent_uuid: "f2806c613ed14dfebb8f027fdedf25b1" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 38643 } } peers { permanent_uuid: "aa01e66b3b06413ca1b2f35391b70c78" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 34953 } }
I20260809 08:56:02.251453 4223 catalog_manager.cc:5495] T 8b0c4e33ee274476a9e85c0b3110f7bc P f2806c613ed14dfebb8f027fdedf25b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2806c613ed14dfebb8f027fdedf25b1 (127.8.154.1). New cstate: current_term: 1 leader_uuid: "f2806c613ed14dfebb8f027fdedf25b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "63dd09310b3b4d9690ada36e8add1807" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 35601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2806c613ed14dfebb8f027fdedf25b1" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 38643 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa01e66b3b06413ca1b2f35391b70c78" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 34953 } health_report { overall_health: UNKNOWN } } }
W20260809 08:56:02.253638 4897 tablet.cc:2369] T 8b0c4e33ee274476a9e85c0b3110f7bc P 63dd09310b3b4d9690ada36e8add1807: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 08:56:02.254843 4504 tablet.cc:2369] T 8b0c4e33ee274476a9e85c0b3110f7bc P f2806c613ed14dfebb8f027fdedf25b1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:56:02.308748 4908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:02.306277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42130 (local address 127.8.154.1:38643)
0809 08:56:02.306349 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:02.306351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:02.306362 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:56:02.306414 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:02.306414 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:02.306434 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:02.306510 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:56:02.306653 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:02.307151 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:02.307933 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:02.308149 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:02.308225 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:02.308382 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:56:02.308490 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:56:02.308592 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:02.308610 (+ 18us) server_negotiation.cc:299] Negotiation successful
0809 08:56:02.308632 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 08:56:02.310586 4828 raft_consensus.cc:1261] T 8b0c4e33ee274476a9e85c0b3110f7bc P 63dd09310b3b4d9690ada36e8add1807 [term 1 FOLLOWER]: Refusing update from remote peer f2806c613ed14dfebb8f027fdedf25b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:56:02.310586 4633 raft_consensus.cc:1261] T 8b0c4e33ee274476a9e85c0b3110f7bc P aa01e66b3b06413ca1b2f35391b70c78 [term 1 FOLLOWER]: Refusing update from remote peer f2806c613ed14dfebb8f027fdedf25b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:56:02.310986 4923 consensus_queue.cc:1035] T 8b0c4e33ee274476a9e85c0b3110f7bc P f2806c613ed14dfebb8f027fdedf25b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "63dd09310b3b4d9690ada36e8add1807" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 35601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:56:02.311128 4926 consensus_queue.cc:1035] T 8b0c4e33ee274476a9e85c0b3110f7bc P f2806c613ed14dfebb8f027fdedf25b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa01e66b3b06413ca1b2f35391b70c78" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 34953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:56:02.313967 4931 mvcc.cc:205] Tried to move back new op lower bound from 7316544562421374976 to 7316544562179215360. Current Snapshot: MvccSnapshot[applied={T|T < 7316544562421374976}]
I20260809 08:56:02.316108 4223 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:56:02.316200 4223 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:56:02.316962 4223 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 53e9822cd6054a2bbf0c5e5409e1fe2f: Sending DeleteTablet for 3 replicas of tablet 8b0c4e33ee274476a9e85c0b3110f7bc
I20260809 08:56:02.317287 4403 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b0c4e33ee274476a9e85c0b3110f7bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:56:02 UTC) from {username='slave'} at 127.0.0.1:42116
I20260809 08:56:02.317324 4604 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b0c4e33ee274476a9e85c0b3110f7bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:56:02 UTC) from {username='slave'} at 127.0.0.1:36358
I20260809 08:56:02.317551 4947 tablet_replica.cc:330] T 8b0c4e33ee274476a9e85c0b3110f7bc P aa01e66b3b06413ca1b2f35391b70c78: stopping tablet replica
I20260809 08:56:02.317540 4946 tablet_replica.cc:330] T 8b0c4e33ee274476a9e85c0b3110f7bc P f2806c613ed14dfebb8f027fdedf25b1: stopping tablet replica
I20260809 08:56:02.317656 4947 raft_consensus.cc:2229] T 8b0c4e33ee274476a9e85c0b3110f7bc P aa01e66b3b06413ca1b2f35391b70c78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:56:02.317736 4946 raft_consensus.cc:2229] T 8b0c4e33ee274476a9e85c0b3110f7bc P f2806c613ed14dfebb8f027fdedf25b1 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:56:02.317719 4796 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b0c4e33ee274476a9e85c0b3110f7bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:56:02 UTC) from {username='slave'} at 127.0.0.1:34852
I20260809 08:56:02.317766 4947 raft_consensus.cc:2258] T 8b0c4e33ee274476a9e85c0b3110f7bc P aa01e66b3b06413ca1b2f35391b70c78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:56:02.317919 4946 raft_consensus.cc:2258] T 8b0c4e33ee274476a9e85c0b3110f7bc P f2806c613ed14dfebb8f027fdedf25b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:56:02.318209 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 4305
I20260809 08:56:02.318230 4947 ts_tablet_manager.cc:1905] T 8b0c4e33ee274476a9e85c0b3110f7bc P aa01e66b3b06413ca1b2f35391b70c78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:56:02.319805 4947 ts_tablet_manager.cc:1918] T 8b0c4e33ee274476a9e85c0b3110f7bc P aa01e66b3b06413ca1b2f35391b70c78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:56:02.319902 4947 log.cc:1196] T 8b0c4e33ee274476a9e85c0b3110f7bc P aa01e66b3b06413ca1b2f35391b70c78: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-1/wal/wals/8b0c4e33ee274476a9e85c0b3110f7bc
I20260809 08:56:02.320240 4947 ts_tablet_manager.cc:1939] T 8b0c4e33ee274476a9e85c0b3110f7bc P aa01e66b3b06413ca1b2f35391b70c78: Deleting consensus metadata
I20260809 08:56:02.320602 4191 catalog_manager.cc:4841] TS aa01e66b3b06413ca1b2f35391b70c78 (127.8.154.2:34953): tablet 8b0c4e33ee274476a9e85c0b3110f7bc (table test-table [id=dfb81ae50fc54c389abcdcb0e39dffb1]) successfully deleted
I20260809 08:56:02.329788 4948 tablet_replica.cc:330] T 8b0c4e33ee274476a9e85c0b3110f7bc P 63dd09310b3b4d9690ada36e8add1807: stopping tablet replica
I20260809 08:56:02.329931 4948 raft_consensus.cc:2229] T 8b0c4e33ee274476a9e85c0b3110f7bc P 63dd09310b3b4d9690ada36e8add1807 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:56:02.330067 4948 raft_consensus.cc:2258] T 8b0c4e33ee274476a9e85c0b3110f7bc P 63dd09310b3b4d9690ada36e8add1807 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:56:02.330502 4948 ts_tablet_manager.cc:1905] T 8b0c4e33ee274476a9e85c0b3110f7bc P 63dd09310b3b4d9690ada36e8add1807: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:56:02.331928 4948 ts_tablet_manager.cc:1918] T 8b0c4e33ee274476a9e85c0b3110f7bc P 63dd09310b3b4d9690ada36e8add1807: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:56:02.331991 4948 log.cc:1196] T 8b0c4e33ee274476a9e85c0b3110f7bc P 63dd09310b3b4d9690ada36e8add1807: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265519942966-8808-0/minicluster-data/ts-2/wal/wals/8b0c4e33ee274476a9e85c0b3110f7bc
I20260809 08:56:02.332273 4948 ts_tablet_manager.cc:1939] T 8b0c4e33ee274476a9e85c0b3110f7bc P 63dd09310b3b4d9690ada36e8add1807: Deleting consensus metadata
I20260809 08:56:02.332674 4193 catalog_manager.cc:4841] TS 63dd09310b3b4d9690ada36e8add1807 (127.8.154.3:35601): tablet 8b0c4e33ee274476a9e85c0b3110f7bc (table test-table [id=dfb81ae50fc54c389abcdcb0e39dffb1]) successfully deleted
W20260809 08:56:02.334429 4192 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.154.1:38643 (error 108)
I20260809 08:56:02.334939 4912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:02.334797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:38643 (local address 127.0.0.1:42134)
0809 08:56:02.334860 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:02.334906 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.1:38643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 08:56:02.334971 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 4506
W20260809 08:56:02.335090 4192 catalog_manager.cc:4584] TS f2806c613ed14dfebb8f027fdedf25b1 (127.8.154.1:38643): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8b0c4e33ee274476a9e85c0b3110f7bc: Network error: Client connection negotiation failed: client connection to 127.8.154.1:38643: connect: Connection refused (error 111)
I20260809 08:56:02.345052 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 4701
I20260809 08:56:02.354423 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 4164
2026-08-09T08:56:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (910 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:56:02 dist-test-slave-t3q3 krb5kdc[4953](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:56:02 dist-test-slave-t3q3 krb5kdc[4953](info): set up 2 sockets
Aug 09 08:56:02 dist-test-slave-t3q3 krb5kdc[4953](info): commencing operation
krb5kdc: starting...
W20260809 08:56:03.626649 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.250s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:56:03 dist-test-slave-t3q3 krb5kdc[4953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:56:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:56:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
I20260809 08:56:03.750797 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:34763
--webserver_interface=127.8.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:46103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:34763
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:56:03.834656 4969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:03.834877 4969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:03.834900 4969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:03.835991 4969 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:56:03.836026 4969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:03.836040 4969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:03.836053 4969 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:56:03.836133 4969 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:56:03.837661 4969 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:46103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:34763
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:34763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:03.837868 4969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:03.838058 4969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:03.842952 4977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:03.842941 4980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:03.843184 4969 server_base.cc:1034] running on GCE node
W20260809 08:56:03.842941 4978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:03.843560 4969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:03.843873 4969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:03.845050 4969 hybrid_clock.cc:648] HybridClock initialized: now 1786265763845041 us; error 35 us; skew 500 ppm
Aug 09 08:56:03 dist-test-slave-t3q3 krb5kdc[4953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265763, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:56:03.851866 4969 init.cc:377] Logged in from keytab as kudu/127.8.154.62@KRBTEST.COM (short username kudu)
I20260809 08:56:03.852427 4969 webserver.cc:460] Webserver started at http://127.8.154.62:44105/ using document root <none> and password file <none>
I20260809 08:56:03.852690 4969 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:03.852762 4969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:03.852911 4969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:03.854029 4969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "04800fcafe48437ea2c4e20605cbc460"
format_stamp: "Formatted at 2026-08-09 08:56:03 on dist-test-slave-t3q3"
server_key: "0d7d34cf76566278c5df6f9fcbc12d30"
server_key_iv: "e686c28eb878d2a5733d6149be397428"
server_key_version: "encryptionkey@0"
I20260809 08:56:03.854410 4969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "04800fcafe48437ea2c4e20605cbc460"
format_stamp: "Formatted at 2026-08-09 08:56:03 on dist-test-slave-t3q3"
server_key: "0d7d34cf76566278c5df6f9fcbc12d30"
server_key_iv: "e686c28eb878d2a5733d6149be397428"
server_key_version: "encryptionkey@0"
I20260809 08:56:03.855803 4969 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:56:03.856614 4990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:03.856791 4969 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:03.856889 4969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "04800fcafe48437ea2c4e20605cbc460"
format_stamp: "Formatted at 2026-08-09 08:56:03 on dist-test-slave-t3q3"
server_key: "0d7d34cf76566278c5df6f9fcbc12d30"
server_key_iv: "e686c28eb878d2a5733d6149be397428"
server_key_version: "encryptionkey@0"
I20260809 08:56:03.856959 4969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:03.869372 4969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:03.871603 4969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:03.871758 4969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:03.875698 4969 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:34763
I20260809 08:56:03.875752 5074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:34763 every 8 connection(s)
I20260809 08:56:03.876179 4969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:56:03.877115 5075 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:03.879428 5075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04800fcafe48437ea2c4e20605cbc460: Bootstrap starting.
I20260809 08:56:03.880040 5075 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04800fcafe48437ea2c4e20605cbc460: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:03.880359 5075 log.cc:824] T 00000000000000000000000000000000 P 04800fcafe48437ea2c4e20605cbc460: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:03.881151 5075 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04800fcafe48437ea2c4e20605cbc460: No bootstrap required, opened a new log
I20260809 08:56:03.882779 5075 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04800fcafe48437ea2c4e20605cbc460 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04800fcafe48437ea2c4e20605cbc460" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 34763 } }
I20260809 08:56:03.882930 5075 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04800fcafe48437ea2c4e20605cbc460 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:03.882953 5075 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04800fcafe48437ea2c4e20605cbc460 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04800fcafe48437ea2c4e20605cbc460, State: Initialized, Role: FOLLOWER
I20260809 08:56:03.883121 5075 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04800fcafe48437ea2c4e20605cbc460 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04800fcafe48437ea2c4e20605cbc460" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 34763 } }
I20260809 08:56:03.883189 5075 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04800fcafe48437ea2c4e20605cbc460 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:56:03.883211 5075 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04800fcafe48437ea2c4e20605cbc460 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:56:03.883261 5075 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04800fcafe48437ea2c4e20605cbc460 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:03.883822 5075 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04800fcafe48437ea2c4e20605cbc460 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04800fcafe48437ea2c4e20605cbc460" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 34763 } }
I20260809 08:56:03.883913 5075 leader_election.cc:302] T 00000000000000000000000000000000 P 04800fcafe48437ea2c4e20605cbc460 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04800fcafe48437ea2c4e20605cbc460; no voters:
I20260809 08:56:03.884152 5075 leader_election.cc:288] T 00000000000000000000000000000000 P 04800fcafe48437ea2c4e20605cbc460 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:56:03.884205 5079 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04800fcafe48437ea2c4e20605cbc460 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:56:03.884395 5079 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04800fcafe48437ea2c4e20605cbc460 [term 1 LEADER]: Becoming Leader. State: Replica: 04800fcafe48437ea2c4e20605cbc460, State: Running, Role: LEADER
I20260809 08:56:03.884438 5075 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04800fcafe48437ea2c4e20605cbc460 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:56:03.884533 5079 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04800fcafe48437ea2c4e20605cbc460 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04800fcafe48437ea2c4e20605cbc460" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 34763 } }
I20260809 08:56:03.884964 5082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04800fcafe48437ea2c4e20605cbc460 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04800fcafe48437ea2c4e20605cbc460. Latest consensus state: current_term: 1 leader_uuid: "04800fcafe48437ea2c4e20605cbc460" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04800fcafe48437ea2c4e20605cbc460" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 34763 } } }
I20260809 08:56:03.885111 5080 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04800fcafe48437ea2c4e20605cbc460 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04800fcafe48437ea2c4e20605cbc460" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04800fcafe48437ea2c4e20605cbc460" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 34763 } } }
I20260809 08:56:03.885161 5082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04800fcafe48437ea2c4e20605cbc460 [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:03.885210 5080 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04800fcafe48437ea2c4e20605cbc460 [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:03.885586 5092 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:56:03.886106 5092 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:56:03.886260 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 4969
I20260809 08:56:03.886394 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:56:03.886623 8808 external_mini_cluster.cc:1428] Setting key 27571ee55c7c4852eff545b5e1eb071a
I20260809 08:56:03.887881 5092 catalog_manager.cc:1347] Generated new cluster ID: 91564c1289824dcf9b30ec68bb6b94fc
I20260809 08:56:03.887941 5092 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 08:56:03 dist-test-slave-t3q3 krb5kdc[4953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265763, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:56:03.899443 5105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:03.887154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55300 (local address 127.8.154.62:34763)
0809 08:56:03.887434 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:03.887445 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:03.887481 (+ 36us) server_negotiation.cc:407] Connection header received
0809 08:56:03.887611 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:03.887620 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:03.887678 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:03.888697 (+ 1019us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:03.888795 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:03.890207 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:03.891378 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:03.892103 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:03.892235 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:03.895187 (+ 2952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:03.895192 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:03.895193 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:03.895206 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:03.897328 (+ 2122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:03.897744 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:03.897747 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:03.897747 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:03.897855 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:03.898065 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:03.898066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:03.898066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:03.898404 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:03.898557 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:03.899023 (+ 466us) server_negotiation.cc:299] Negotiation successful
0809 08:56:03.899175 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":96,"threads_started":1}
I20260809 08:56:03.900092 5092 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:56:03.900683 5092 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:56:03.908403 5092 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04800fcafe48437ea2c4e20605cbc460: Generated new TSK 0
I20260809 08:56:03.908612 5092 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab.
Entry for principal kudu/127.8.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab.
I20260809 08:56:03.940878 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.1:0
--local_ip_for_outbound_sockets=127.8.154.1
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:34763
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:46103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:56:04.029793 5110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:04.030006 5110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:04.030031 5110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:04.031097 5110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:04.031131 5110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:04.031157 5110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.1
I20260809 08:56:04.032542 5110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:46103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:34763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.1
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:04.032752 5110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:04.032992 5110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:04.038239 5118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:04.038251 5120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:04.038354 5122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:04.038923 5110 server_base.cc:1034] running on GCE node
I20260809 08:56:04.039232 5110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:04.039605 5110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:04.040784 5110 hybrid_clock.cc:648] HybridClock initialized: now 1786265764040753 us; error 48 us; skew 500 ppm
Aug 09 08:56:04 dist-test-slave-t3q3 krb5kdc[4953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265764, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:56:04.043877 5110 init.cc:377] Logged in from keytab as kudu/127.8.154.1@KRBTEST.COM (short username kudu)
I20260809 08:56:04.044440 5110 webserver.cc:460] Webserver started at http://127.8.154.1:43753/ using document root <none> and password file <none>
I20260809 08:56:04.044687 5110 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:04.044734 5110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:04.044888 5110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:04.046000 5110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-0/data/instance:
uuid: "d5deb73e487847b5b25c85285c1038b2"
format_stamp: "Formatted at 2026-08-09 08:56:04 on dist-test-slave-t3q3"
server_key: "d8fdcc84885605b0e3a952a6cd862e81"
server_key_iv: "7aed7a4c958167e5aeacc4cbda9895fe"
server_key_version: "encryptionkey@0"
I20260809 08:56:04.046384 5110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance:
uuid: "d5deb73e487847b5b25c85285c1038b2"
format_stamp: "Formatted at 2026-08-09 08:56:04 on dist-test-slave-t3q3"
server_key: "d8fdcc84885605b0e3a952a6cd862e81"
server_key_iv: "7aed7a4c958167e5aeacc4cbda9895fe"
server_key_version: "encryptionkey@0"
I20260809 08:56:04.048345 5110 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260809 08:56:04.049190 5130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:04.049382 5110 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 08:56:04.049477 5110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-0/wal
uuid: "d5deb73e487847b5b25c85285c1038b2"
format_stamp: "Formatted at 2026-08-09 08:56:04 on dist-test-slave-t3q3"
server_key: "d8fdcc84885605b0e3a952a6cd862e81"
server_key_iv: "7aed7a4c958167e5aeacc4cbda9895fe"
server_key_version: "encryptionkey@0"
I20260809 08:56:04.049556 5110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:04.065197 5110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:04.067310 5110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:04.067473 5110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:04.067730 5110 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:56:04.068115 5110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:56:04.068156 5110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:04.068179 5110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:56:04.068192 5110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:04.074370 5110 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.1:39335
I20260809 08:56:04.074442 5303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.1:39335 every 8 connection(s)
I20260809 08:56:04.074761 5110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
Aug 09 08:56:04 dist-test-slave-t3q3 krb5kdc[4953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265764, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:56:04.083638 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 5110
I20260809 08:56:04.083784 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance
I20260809 08:56:04.084025 8808 external_mini_cluster.cc:1428] Setting key f2d7e6aea27c2f9ac983788ce7ac04ab
I20260809 08:56:04.084503 5105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:04.075537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:36827 (local address 127.8.154.62:34763)
0809 08:56:04.075599 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:04.075601 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:04.075845 (+ 244us) server_negotiation.cc:407] Connection header received
0809 08:56:04.076401 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:04.076402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:04.076416 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:04.076482 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:04.077427 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.077945 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.079023 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.079237 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.079302 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:04.081561 (+ 2259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:04.081563 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:04.081564 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:04.081570 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:04.082969 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:04.083338 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:04.083339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:04.083339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:04.083390 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:04.083690 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:04.083690 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:04.083691 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:04.083863 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:04.083924 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:04.084310 (+ 386us) server_negotiation.cc:299] Negotiation successful
0809 08:56:04.084407 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 08:56:04.084779 5306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:04.075511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:34763 (local address 127.8.154.1:36827)
0809 08:56:04.075689 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:04.075732 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:04.076307 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:04.076544 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:04.076548 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:04.077131 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:56:04.077332 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:04.077337 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.078060 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.078061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:04.078927 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:04.078930 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.079342 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.079343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:04.079461 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:04.079957 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:56:04.079965 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:56:04.081465 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:56:04.083044 (+ 1579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:04.083050 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:04.083052 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:04.083273 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:04.083474 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:04.083475 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:04.083475 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:04.083622 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:04.083924 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:56:04.083932 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:56:04.084148 (+ 216us) client_negotiation.cc:769] Sending connection context
0809 08:56:04.084271 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 08:56:04.084447 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":97,"threads_started":1}
I20260809 08:56:04.085301 5304 heartbeater.cc:347] Connected to a master server at 127.8.154.62:34763
I20260809 08:56:04.085403 5304 heartbeater.cc:464] Registering TS with master...
I20260809 08:56:04.085587 5304 heartbeater.cc:511] Master 127.8.154.62:34763 requested a full tablet report, sending...
I20260809 08:56:04.086166 5016 ts_manager.cc:194] Registered new tserver with Master: d5deb73e487847b5b25c85285c1038b2 (127.8.154.1:39335)
I20260809 08:56:04.086994 5016 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.1@KRBTEST.COM'} at 127.8.154.1:36827
WARNING: no policy specified for kudu/127.8.154.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab.
Entry for principal kudu/127.8.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab.
I20260809 08:56:04.110807 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.2:0
--local_ip_for_outbound_sockets=127.8.154.2
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:34763
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:46103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:56:04.196959 5309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:04.197182 5309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:04.197207 5309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:04.198315 5309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:04.198354 5309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:04.198380 5309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.2
I20260809 08:56:04.199846 5309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:46103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:34763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.2
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:04.200133 5309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:04.200342 5309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:04.205719 5319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:04.205731 5315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:04.205739 5314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:04.205993 5309 server_base.cc:1034] running on GCE node
I20260809 08:56:04.206246 5309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:04.206506 5309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:04.207702 5309 hybrid_clock.cc:648] HybridClock initialized: now 1786265764207703 us; error 47 us; skew 500 ppm
Aug 09 08:56:04 dist-test-slave-t3q3 krb5kdc[4953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265764, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:56:04.210492 5309 init.cc:377] Logged in from keytab as kudu/127.8.154.2@KRBTEST.COM (short username kudu)
I20260809 08:56:04.210983 5309 webserver.cc:460] Webserver started at http://127.8.154.2:41129/ using document root <none> and password file <none>
I20260809 08:56:04.211234 5309 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:04.211303 5309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:04.211449 5309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:04.212625 5309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-1/data/instance:
uuid: "39657962b3284ed7aeb1b5299a74fae4"
format_stamp: "Formatted at 2026-08-09 08:56:04 on dist-test-slave-t3q3"
server_key: "2c3627603413dde1084478dcbedc07aa"
server_key_iv: "e335cdb4fae18dd120748ebc907dda6c"
server_key_version: "encryptionkey@0"
I20260809 08:56:04.213021 5309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance:
uuid: "39657962b3284ed7aeb1b5299a74fae4"
format_stamp: "Formatted at 2026-08-09 08:56:04 on dist-test-slave-t3q3"
server_key: "2c3627603413dde1084478dcbedc07aa"
server_key_iv: "e335cdb4fae18dd120748ebc907dda6c"
server_key_version: "encryptionkey@0"
I20260809 08:56:04.214442 5309 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:56:04.215250 5326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:04.215425 5309 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:56:04.215543 5309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-1/wal
uuid: "39657962b3284ed7aeb1b5299a74fae4"
format_stamp: "Formatted at 2026-08-09 08:56:04 on dist-test-slave-t3q3"
server_key: "2c3627603413dde1084478dcbedc07aa"
server_key_iv: "e335cdb4fae18dd120748ebc907dda6c"
server_key_version: "encryptionkey@0"
I20260809 08:56:04.215632 5309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:04.245728 5309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:04.247761 5309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:04.247879 5309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:04.248139 5309 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:56:04.248442 5309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:56:04.248474 5309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:04.248498 5309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:56:04.248518 5309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:04.254827 5309 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.2:38653
I20260809 08:56:04.254873 5513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.2:38653 every 8 connection(s)
I20260809 08:56:04.255209 5309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
I20260809 08:56:04.256407 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 5309
I20260809 08:56:04.256522 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance
I20260809 08:56:04.256716 8808 external_mini_cluster.cc:1428] Setting key 061c0d4a1e39f7cb226e52f694f62d80
Aug 09 08:56:04 dist-test-slave-t3q3 krb5kdc[4953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265764, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:56:04.265133 5105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:04.255966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:44349 (local address 127.8.154.62:34763)
0809 08:56:04.256109 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:04.256112 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:04.256291 (+ 179us) server_negotiation.cc:407] Connection header received
0809 08:56:04.256994 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:04.256996 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:04.257008 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:04.257078 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:04.258115 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.258623 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.259718 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.259921 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.259999 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:04.262233 (+ 2234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:04.262234 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:04.262235 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:04.262243 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:04.263703 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:04.264155 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:04.264156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:04.264156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:04.264215 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:04.264470 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:04.264471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:04.264472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:04.264589 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:04.264664 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:04.264942 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 08:56:04.265033 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260809 08:56:04.265314 5517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:04.255866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:34763 (local address 127.8.154.2:44349)
0809 08:56:04.256121 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:04.256173 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:04.256888 (+ 715us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:04.257164 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:04.257168 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:04.257800 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:56:04.257999 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:04.258004 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.258721 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.258722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:04.259622 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:04.259624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.260006 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.260007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:04.260186 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:04.260733 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:56:04.260742 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:56:04.262136 (+ 1394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:56:04.263805 (+ 1669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:04.263813 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:04.263815 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:04.264043 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:04.264282 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:04.264283 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:04.264283 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:04.264396 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:04.264655 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:56:04.264663 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:56:04.264807 (+ 144us) client_negotiation.cc:769] Sending connection context
0809 08:56:04.264907 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 08:56:04.265072 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":111,"threads_started":1}
I20260809 08:56:04.265805 5514 heartbeater.cc:347] Connected to a master server at 127.8.154.62:34763
WARNING: no policy specified for kudu/127.8.154.3@KRBTEST.COM; defaulting to no policy
I20260809 08:56:04.265903 5514 heartbeater.cc:464] Registering TS with master...
I20260809 08:56:04.266105 5514 heartbeater.cc:511] Master 127.8.154.62:34763 requested a full tablet report, sending...
I20260809 08:56:04.266539 5016 ts_manager.cc:194] Registered new tserver with Master: 39657962b3284ed7aeb1b5299a74fae4 (127.8.154.2:38653)
I20260809 08:56:04.267089 5016 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.2@KRBTEST.COM'} at 127.8.154.2:44349
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab.
Entry for principal kudu/127.8.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab.
I20260809 08:56:04.284610 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.3:0
--local_ip_for_outbound_sockets=127.8.154.3
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:34763
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:46103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:56:04.370237 5521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:04.370426 5521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:04.370450 5521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:04.371537 5521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:04.371574 5521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:04.371606 5521 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.3
I20260809 08:56:04.373090 5521 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:46103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:34763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5521
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.3
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:04.373278 5521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:04.373459 5521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:04.378574 5536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:04.378594 5534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:04.378582 5530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:04.378687 5521 server_base.cc:1034] running on GCE node
I20260809 08:56:04.378973 5521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:04.379247 5521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:04.380399 5521 hybrid_clock.cc:648] HybridClock initialized: now 1786265764380396 us; error 39 us; skew 500 ppm
Aug 09 08:56:04 dist-test-slave-t3q3 krb5kdc[4953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265764, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:56:04.382941 5521 init.cc:377] Logged in from keytab as kudu/127.8.154.3@KRBTEST.COM (short username kudu)
I20260809 08:56:04.383358 5521 webserver.cc:460] Webserver started at http://127.8.154.3:35611/ using document root <none> and password file <none>
I20260809 08:56:04.383594 5521 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:04.383664 5521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:04.383814 5521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:04.385022 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-2/data/instance:
uuid: "14b5d72bacc54d1d93b06b7135e58a13"
format_stamp: "Formatted at 2026-08-09 08:56:04 on dist-test-slave-t3q3"
server_key: "5581da9dc20151910d03cf400e3bba4f"
server_key_iv: "dc745c04559cae41a21206433444cc4a"
server_key_version: "encryptionkey@0"
I20260809 08:56:04.385375 5521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance:
uuid: "14b5d72bacc54d1d93b06b7135e58a13"
format_stamp: "Formatted at 2026-08-09 08:56:04 on dist-test-slave-t3q3"
server_key: "5581da9dc20151910d03cf400e3bba4f"
server_key_iv: "dc745c04559cae41a21206433444cc4a"
server_key_version: "encryptionkey@0"
I20260809 08:56:04.386796 5521 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 08:56:04.387514 5545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:04.387681 5521 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:56:04.387778 5521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-2/wal
uuid: "14b5d72bacc54d1d93b06b7135e58a13"
format_stamp: "Formatted at 2026-08-09 08:56:04 on dist-test-slave-t3q3"
server_key: "5581da9dc20151910d03cf400e3bba4f"
server_key_iv: "dc745c04559cae41a21206433444cc4a"
server_key_version: "encryptionkey@0"
I20260809 08:56:04.387856 5521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:04.427007 5521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:04.429088 5521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:04.429212 5521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:04.429395 5521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:56:04.429723 5521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:56:04.429757 5521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:04.429800 5521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:56:04.429826 5521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:04.435714 5521 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.3:46765
I20260809 08:56:04.435796 5719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.3:46765 every 8 connection(s)
I20260809 08:56:04.436192 5521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
I20260809 08:56:04.441339 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 5521
I20260809 08:56:04.441480 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance
I20260809 08:56:04.441727 8808 external_mini_cluster.cc:1428] Setting key 7fabf0b7e82b7bbb2729e56a24119065
Aug 09 08:56:04 dist-test-slave-t3q3 krb5kdc[4953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265764, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:56:04.445781 5105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:04.437003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:58797 (local address 127.8.154.62:34763)
0809 08:56:04.437068 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:04.437071 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:04.437282 (+ 211us) server_negotiation.cc:407] Connection header received
0809 08:56:04.437729 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:04.437730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:04.437743 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:04.437801 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:04.438737 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.439245 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.440330 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.440548 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.440614 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:04.443296 (+ 2682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:04.443297 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:04.443297 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:04.443304 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:04.444634 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:04.445001 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:04.445002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:04.445002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:04.445052 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:04.445277 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:04.445278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:04.445278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:04.445378 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:04.445428 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:04.445656 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 08:56:04.445718 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 08:56:04.446002 5723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:04.436904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:34763 (local address 127.8.154.3:58797)
0809 08:56:04.437099 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:04.437150 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:04.437642 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:04.437863 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:04.437867 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:04.438452 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:56:04.438657 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:04.438662 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.439326 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.439327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:04.440227 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:04.440230 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.440637 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.440638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:04.440752 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:04.441479 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:56:04.441489 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:56:04.443210 (+ 1721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:56:04.444704 (+ 1494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:04.444710 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:04.444712 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:04.444949 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:04.445122 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:04.445123 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:04.445123 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:04.445225 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:04.445428 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:56:04.445432 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:56:04.445545 (+ 113us) client_negotiation.cc:769] Sending connection context
0809 08:56:04.445630 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 08:56:04.445749 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":100,"threads_started":1}
I20260809 08:56:04.446451 5721 heartbeater.cc:347] Connected to a master server at 127.8.154.62:34763
I20260809 08:56:04.446548 5721 heartbeater.cc:464] Registering TS with master...
I20260809 08:56:04.446761 5721 heartbeater.cc:511] Master 127.8.154.62:34763 requested a full tablet report, sending...
I20260809 08:56:04.447134 5016 ts_manager.cc:194] Registered new tserver with Master: 14b5d72bacc54d1d93b06b7135e58a13 (127.8.154.3:46765)
I20260809 08:56:04.447503 5016 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.3@KRBTEST.COM'} at 127.8.154.3:58797
I20260809 08:56:04.453727 8808 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:56:04.461684 5105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:04.455128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55314 (local address 127.8.154.62:34763)
0809 08:56:04.455170 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:04.455171 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:04.455351 (+ 180us) server_negotiation.cc:407] Connection header received
0809 08:56:04.455395 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:04.455396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:04.455406 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:04.455463 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:04.455980 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.456579 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.457315 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.457519 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.457578 (+ 59us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:04.458827 (+ 1249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:04.458829 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:04.458829 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:04.458836 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:04.460543 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:04.460993 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:04.460994 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:04.460994 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:04.461062 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:04.461246 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:04.461246 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:04.461247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:04.461376 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:04.461459 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:04.461563 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 08:56:04.461619 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 08:56:04.463430 5016 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55314:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:56:04.465103 5016 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:56:04.472096 5745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:04.467651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:46765 (local address 127.0.0.1:42276)
0809 08:56:04.468040 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:04.468121 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:04.468203 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:04.468561 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:04.468565 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:04.468568 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:56:04.468656 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:04.468658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.469685 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.469686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:04.470825 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:04.470828 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.471752 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.471754 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:04.471836 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:04.471854 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 08:56:04.471925 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 08:56:04.471985 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":66,"threads_started":1}
I20260809 08:56:04.472235 5749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:04.468120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33150 (local address 127.8.154.1:39335)
0809 08:56:04.468342 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:04.468347 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:04.468357 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:56:04.468403 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:04.468406 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:04.468422 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:04.468506 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:56:04.468909 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.470076 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.471283 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.471816 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.471893 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:04.471955 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:04.472085 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 08:56:04.472137 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":97,"threads_started":1}
I20260809 08:56:04.472551 5747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:04.468052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49728 (local address 127.8.154.2:38653)
0809 08:56:04.468685 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:04.468690 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:04.468702 (+ 12us) server_negotiation.cc:407] Connection header received
0809 08:56:04.468775 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:04.468777 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:04.468792 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:04.468877 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:56:04.469027 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.470101 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.471373 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.472108 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.472167 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:04.472219 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:04.472418 (+ 199us) server_negotiation.cc:299] Negotiation successful
0809 08:56:04.472467 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":127,"threads_started":1}
I20260809 08:56:04.472715 5744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:04.467839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42276 (local address 127.8.154.3:46765)
0809 08:56:04.468371 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:04.468374 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:04.468382 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:56:04.468419 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:04.468421 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:04.468434 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:04.468539 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:56:04.468747 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.469570 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.470926 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.471656 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.471730 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:04.471787 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:04.472513 (+ 726us) server_negotiation.cc:299] Negotiation successful
0809 08:56:04.472580 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":94,"threads_started":1}
I20260809 08:56:04.472211 5741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:04.467651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:39335 (local address 127.0.0.1:33150)
0809 08:56:04.468174 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:04.468187 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:04.468246 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:04.468766 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:04.468766 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:04.468768 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:56:04.468823 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:04.468825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.470193 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.470194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:04.471192 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:04.471194 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.471894 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.471895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:04.471960 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:04.471964 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:56:04.472031 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 08:56:04.472098 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"mutex_wait_us":46,"thread_start_us":70,"threads_started":1}
I20260809 08:56:04.473563 5622 tablet_service.cc:1467] Processing CreateTablet for tablet 159a541cd3ca4b9ea80cc9d280f85ee3 (DEFAULT_TABLE table=test-table [id=0620fc2d4fa247fcad7360a9d3968866]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:56:04.473565 5204 tablet_service.cc:1467] Processing CreateTablet for tablet 159a541cd3ca4b9ea80cc9d280f85ee3 (DEFAULT_TABLE table=test-table [id=0620fc2d4fa247fcad7360a9d3968866]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:56:04.473917 5622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 159a541cd3ca4b9ea80cc9d280f85ee3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:04.473917 5204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 159a541cd3ca4b9ea80cc9d280f85ee3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:04.476728 5752 tablet_bootstrap.cc:492] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 14b5d72bacc54d1d93b06b7135e58a13: Bootstrap starting.
I20260809 08:56:04.472468 5743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:04.467873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:38653 (local address 127.0.0.1:49728)
0809 08:56:04.468628 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:04.468639 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:04.468694 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:04.468889 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:04.468889 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:04.468892 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:56:04.468944 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:04.468946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.470210 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.470211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:04.471275 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:04.471278 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.472255 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.472255 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:04.472331 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:04.472336 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:56:04.472391 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 08:56:04.472426 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":733,"thread_start_us":66,"threads_started":1}
I20260809 08:56:04.477466 5752 tablet_bootstrap.cc:654] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 14b5d72bacc54d1d93b06b7135e58a13: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:04.477526 5407 tablet_service.cc:1467] Processing CreateTablet for tablet 159a541cd3ca4b9ea80cc9d280f85ee3 (DEFAULT_TABLE table=test-table [id=0620fc2d4fa247fcad7360a9d3968866]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:56:04.477880 5407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 159a541cd3ca4b9ea80cc9d280f85ee3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:04.478030 5752 log.cc:824] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 14b5d72bacc54d1d93b06b7135e58a13: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:04.478389 5751 tablet_bootstrap.cc:492] T 159a541cd3ca4b9ea80cc9d280f85ee3 P d5deb73e487847b5b25c85285c1038b2: Bootstrap starting.
I20260809 08:56:04.478825 5752 tablet_bootstrap.cc:492] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 14b5d72bacc54d1d93b06b7135e58a13: No bootstrap required, opened a new log
I20260809 08:56:04.478900 5752 ts_tablet_manager.cc:1397] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 14b5d72bacc54d1d93b06b7135e58a13: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:56:04.479092 5751 tablet_bootstrap.cc:654] T 159a541cd3ca4b9ea80cc9d280f85ee3 P d5deb73e487847b5b25c85285c1038b2: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:04.479869 5751 log.cc:824] T 159a541cd3ca4b9ea80cc9d280f85ee3 P d5deb73e487847b5b25c85285c1038b2: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:04.480700 5751 tablet_bootstrap.cc:492] T 159a541cd3ca4b9ea80cc9d280f85ee3 P d5deb73e487847b5b25c85285c1038b2: No bootstrap required, opened a new log
I20260809 08:56:04.480775 5751 ts_tablet_manager.cc:1397] T 159a541cd3ca4b9ea80cc9d280f85ee3 P d5deb73e487847b5b25c85285c1038b2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260809 08:56:04.481076 5752 raft_consensus.cc:348] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 14b5d72bacc54d1d93b06b7135e58a13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5deb73e487847b5b25c85285c1038b2" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 39335 } } peers { permanent_uuid: "14b5d72bacc54d1d93b06b7135e58a13" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 46765 } } peers { permanent_uuid: "39657962b3284ed7aeb1b5299a74fae4" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38653 } }
I20260809 08:56:04.481280 5752 raft_consensus.cc:374] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 14b5d72bacc54d1d93b06b7135e58a13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:04.481316 5752 raft_consensus.cc:729] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 14b5d72bacc54d1d93b06b7135e58a13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14b5d72bacc54d1d93b06b7135e58a13, State: Initialized, Role: FOLLOWER
I20260809 08:56:04.481433 5752 consensus_queue.cc:260] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 14b5d72bacc54d1d93b06b7135e58a13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5deb73e487847b5b25c85285c1038b2" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 39335 } } peers { permanent_uuid: "14b5d72bacc54d1d93b06b7135e58a13" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 46765 } } peers { permanent_uuid: "39657962b3284ed7aeb1b5299a74fae4" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38653 } }
I20260809 08:56:04.481717 5752 ts_tablet_manager.cc:1428] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 14b5d72bacc54d1d93b06b7135e58a13: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:56:04.481731 5721 heartbeater.cc:503] Master 127.8.154.62:34763 was elected leader, sending a full tablet report...
I20260809 08:56:04.481923 5756 tablet_bootstrap.cc:492] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 39657962b3284ed7aeb1b5299a74fae4: Bootstrap starting.
I20260809 08:56:04.482561 5756 tablet_bootstrap.cc:654] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 39657962b3284ed7aeb1b5299a74fae4: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:04.483166 5756 log.cc:824] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 39657962b3284ed7aeb1b5299a74fae4: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:04.483448 5751 raft_consensus.cc:348] T 159a541cd3ca4b9ea80cc9d280f85ee3 P d5deb73e487847b5b25c85285c1038b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5deb73e487847b5b25c85285c1038b2" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 39335 } } peers { permanent_uuid: "14b5d72bacc54d1d93b06b7135e58a13" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 46765 } } peers { permanent_uuid: "39657962b3284ed7aeb1b5299a74fae4" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38653 } }
I20260809 08:56:04.483621 5751 raft_consensus.cc:374] T 159a541cd3ca4b9ea80cc9d280f85ee3 P d5deb73e487847b5b25c85285c1038b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:04.483666 5751 raft_consensus.cc:729] T 159a541cd3ca4b9ea80cc9d280f85ee3 P d5deb73e487847b5b25c85285c1038b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5deb73e487847b5b25c85285c1038b2, State: Initialized, Role: FOLLOWER
I20260809 08:56:04.483834 5751 consensus_queue.cc:260] T 159a541cd3ca4b9ea80cc9d280f85ee3 P d5deb73e487847b5b25c85285c1038b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5deb73e487847b5b25c85285c1038b2" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 39335 } } peers { permanent_uuid: "14b5d72bacc54d1d93b06b7135e58a13" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 46765 } } peers { permanent_uuid: "39657962b3284ed7aeb1b5299a74fae4" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38653 } }
I20260809 08:56:04.483946 5756 tablet_bootstrap.cc:492] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 39657962b3284ed7aeb1b5299a74fae4: No bootstrap required, opened a new log
I20260809 08:56:04.484020 5756 ts_tablet_manager.cc:1397] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 39657962b3284ed7aeb1b5299a74fae4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:56:04.484165 5751 ts_tablet_manager.cc:1428] T 159a541cd3ca4b9ea80cc9d280f85ee3 P d5deb73e487847b5b25c85285c1038b2: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 08:56:04.484309 5304 heartbeater.cc:503] Master 127.8.154.62:34763 was elected leader, sending a full tablet report...
I20260809 08:56:04.485774 5756 raft_consensus.cc:348] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 39657962b3284ed7aeb1b5299a74fae4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5deb73e487847b5b25c85285c1038b2" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 39335 } } peers { permanent_uuid: "14b5d72bacc54d1d93b06b7135e58a13" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 46765 } } peers { permanent_uuid: "39657962b3284ed7aeb1b5299a74fae4" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38653 } }
I20260809 08:56:04.485927 5756 raft_consensus.cc:374] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 39657962b3284ed7aeb1b5299a74fae4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:04.485967 5756 raft_consensus.cc:729] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 39657962b3284ed7aeb1b5299a74fae4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39657962b3284ed7aeb1b5299a74fae4, State: Initialized, Role: FOLLOWER
I20260809 08:56:04.486078 5756 consensus_queue.cc:260] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 39657962b3284ed7aeb1b5299a74fae4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5deb73e487847b5b25c85285c1038b2" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 39335 } } peers { permanent_uuid: "14b5d72bacc54d1d93b06b7135e58a13" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 46765 } } peers { permanent_uuid: "39657962b3284ed7aeb1b5299a74fae4" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38653 } }
I20260809 08:56:04.486335 5756 ts_tablet_manager.cc:1428] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 39657962b3284ed7aeb1b5299a74fae4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:56:04.486367 5514 heartbeater.cc:503] Master 127.8.154.62:34763 was elected leader, sending a full tablet report...
W20260809 08:56:04.505736 5516 tablet.cc:2369] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 39657962b3284ed7aeb1b5299a74fae4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:56:04.539600 5757 raft_consensus.cc:482] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 14b5d72bacc54d1d93b06b7135e58a13 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:56:04.539758 5757 raft_consensus.cc:504] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 14b5d72bacc54d1d93b06b7135e58a13 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5deb73e487847b5b25c85285c1038b2" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 39335 } } peers { permanent_uuid: "14b5d72bacc54d1d93b06b7135e58a13" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 46765 } } peers { permanent_uuid: "39657962b3284ed7aeb1b5299a74fae4" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38653 } }
I20260809 08:56:04.540140 5757 leader_election.cc:288] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 14b5d72bacc54d1d93b06b7135e58a13 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d5deb73e487847b5b25c85285c1038b2 (127.8.154.1:39335), 39657962b3284ed7aeb1b5299a74fae4 (127.8.154.2:38653)
I20260809 08:56:04.544349 5749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:04.540470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:60433 (local address 127.8.154.1:39335)
0809 08:56:04.540525 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:04.540527 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:04.540538 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:56:04.540575 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:04.540576 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:04.540589 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:04.540649 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:56:04.540932 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.541773 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.543299 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.543918 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.544001 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:04.544013 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:04.544229 (+ 216us) server_negotiation.cc:299] Negotiation successful
0809 08:56:04.544274 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 08:56:04.544349 5747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:04.540662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:47089 (local address 127.8.154.2:38653)
0809 08:56:04.540710 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:04.540712 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:04.540837 (+ 125us) server_negotiation.cc:407] Connection header received
0809 08:56:04.540971 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:04.540972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:04.540988 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:04.541079 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:56:04.541222 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.541902 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.543299 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.543918 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.544001 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:04.544013 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:04.544229 (+ 216us) server_negotiation.cc:299] Negotiation successful
0809 08:56:04.544274 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 08:56:04.544369 5723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:04.540380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:39335 (local address 127.8.154.3:60433)
0809 08:56:04.540443 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:04.540457 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:04.540511 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:04.540713 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:04.540715 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:04.540717 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:56:04.540806 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:04.540828 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.541883 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.541884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:04.543188 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:04.543191 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.544024 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.544025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:04.544152 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:04.544154 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 08:56:04.544209 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 08:56:04.544265 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":20}
I20260809 08:56:04.544378 5762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:04.540495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:38653 (local address 127.8.154.3:47089)
0809 08:56:04.540750 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:04.540761 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:04.540873 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:04.541089 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:04.541089 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:04.541091 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:56:04.541150 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:04.541152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.541953 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.541954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:04.543188 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:04.543191 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.544024 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.544025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:04.544138 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:04.544143 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:56:04.544209 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 08:56:04.544265 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":40,"thread_start_us":162,"threads_started":1}
I20260809 08:56:04.544862 5239 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "159a541cd3ca4b9ea80cc9d280f85ee3" candidate_uuid: "14b5d72bacc54d1d93b06b7135e58a13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5deb73e487847b5b25c85285c1038b2" is_pre_election: true
I20260809 08:56:04.545027 5239 raft_consensus.cc:2454] T 159a541cd3ca4b9ea80cc9d280f85ee3 P d5deb73e487847b5b25c85285c1038b2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14b5d72bacc54d1d93b06b7135e58a13 in term 0.
I20260809 08:56:04.545071 5436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "159a541cd3ca4b9ea80cc9d280f85ee3" candidate_uuid: "14b5d72bacc54d1d93b06b7135e58a13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39657962b3284ed7aeb1b5299a74fae4" is_pre_election: true
I20260809 08:56:04.545248 5436 raft_consensus.cc:2454] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 39657962b3284ed7aeb1b5299a74fae4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14b5d72bacc54d1d93b06b7135e58a13 in term 0.
I20260809 08:56:04.545341 5548 leader_election.cc:302] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 14b5d72bacc54d1d93b06b7135e58a13 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14b5d72bacc54d1d93b06b7135e58a13, d5deb73e487847b5b25c85285c1038b2; no voters:
I20260809 08:56:04.545485 5757 raft_consensus.cc:2781] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 14b5d72bacc54d1d93b06b7135e58a13 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:56:04.545545 5757 raft_consensus.cc:482] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 14b5d72bacc54d1d93b06b7135e58a13 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:56:04.545562 5757 raft_consensus.cc:3037] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 14b5d72bacc54d1d93b06b7135e58a13 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:04.546307 5757 raft_consensus.cc:504] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 14b5d72bacc54d1d93b06b7135e58a13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5deb73e487847b5b25c85285c1038b2" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 39335 } } peers { permanent_uuid: "14b5d72bacc54d1d93b06b7135e58a13" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 46765 } } peers { permanent_uuid: "39657962b3284ed7aeb1b5299a74fae4" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38653 } }
I20260809 08:56:04.546451 5757 leader_election.cc:288] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 14b5d72bacc54d1d93b06b7135e58a13 [CANDIDATE]: Term 1 election: Requested vote from peers d5deb73e487847b5b25c85285c1038b2 (127.8.154.1:39335), 39657962b3284ed7aeb1b5299a74fae4 (127.8.154.2:38653)
I20260809 08:56:04.546598 5239 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "159a541cd3ca4b9ea80cc9d280f85ee3" candidate_uuid: "14b5d72bacc54d1d93b06b7135e58a13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5deb73e487847b5b25c85285c1038b2"
I20260809 08:56:04.546651 5436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "159a541cd3ca4b9ea80cc9d280f85ee3" candidate_uuid: "14b5d72bacc54d1d93b06b7135e58a13" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39657962b3284ed7aeb1b5299a74fae4"
I20260809 08:56:04.546693 5239 raft_consensus.cc:3037] T 159a541cd3ca4b9ea80cc9d280f85ee3 P d5deb73e487847b5b25c85285c1038b2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:04.546718 5436 raft_consensus.cc:3037] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 39657962b3284ed7aeb1b5299a74fae4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:04.547669 5436 raft_consensus.cc:2454] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 39657962b3284ed7aeb1b5299a74fae4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14b5d72bacc54d1d93b06b7135e58a13 in term 1.
I20260809 08:56:04.547668 5239 raft_consensus.cc:2454] T 159a541cd3ca4b9ea80cc9d280f85ee3 P d5deb73e487847b5b25c85285c1038b2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14b5d72bacc54d1d93b06b7135e58a13 in term 1.
I20260809 08:56:04.547896 5548 leader_election.cc:302] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 14b5d72bacc54d1d93b06b7135e58a13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14b5d72bacc54d1d93b06b7135e58a13, 39657962b3284ed7aeb1b5299a74fae4; no voters:
I20260809 08:56:04.547991 5757 raft_consensus.cc:2781] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 14b5d72bacc54d1d93b06b7135e58a13 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:56:04.548200 5757 raft_consensus.cc:686] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 14b5d72bacc54d1d93b06b7135e58a13 [term 1 LEADER]: Becoming Leader. State: Replica: 14b5d72bacc54d1d93b06b7135e58a13, State: Running, Role: LEADER
I20260809 08:56:04.548321 5757 consensus_queue.cc:237] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 14b5d72bacc54d1d93b06b7135e58a13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5deb73e487847b5b25c85285c1038b2" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 39335 } } peers { permanent_uuid: "14b5d72bacc54d1d93b06b7135e58a13" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 46765 } } peers { permanent_uuid: "39657962b3284ed7aeb1b5299a74fae4" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38653 } }
I20260809 08:56:04.549315 5015 catalog_manager.cc:5495] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 14b5d72bacc54d1d93b06b7135e58a13 reported cstate change: term changed from 0 to 1, leader changed from <none> to 14b5d72bacc54d1d93b06b7135e58a13 (127.8.154.3). New cstate: current_term: 1 leader_uuid: "14b5d72bacc54d1d93b06b7135e58a13" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5deb73e487847b5b25c85285c1038b2" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 39335 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14b5d72bacc54d1d93b06b7135e58a13" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 46765 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39657962b3284ed7aeb1b5299a74fae4" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38653 } health_report { overall_health: UNKNOWN } } }
I20260809 08:56:04.564021 5744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:04.561631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42290 (local address 127.8.154.3:46765)
0809 08:56:04.561683 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:04.561685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:04.561722 (+ 37us) server_negotiation.cc:407] Connection header received
0809 08:56:04.561794 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:04.561795 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:04.561819 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:04.561892 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:56:04.562016 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.562536 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.563259 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.563468 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.563553 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:04.563680 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:56:04.563774 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:56:04.563859 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:04.563897 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 08:56:04.563905 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:56:04.565973 5239 raft_consensus.cc:1261] T 159a541cd3ca4b9ea80cc9d280f85ee3 P d5deb73e487847b5b25c85285c1038b2 [term 1 FOLLOWER]: Refusing update from remote peer 14b5d72bacc54d1d93b06b7135e58a13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:56:04.565973 5436 raft_consensus.cc:1261] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 39657962b3284ed7aeb1b5299a74fae4 [term 1 FOLLOWER]: Refusing update from remote peer 14b5d72bacc54d1d93b06b7135e58a13: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:56:04.566329 5757 consensus_queue.cc:1035] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 14b5d72bacc54d1d93b06b7135e58a13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39657962b3284ed7aeb1b5299a74fae4" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38653 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:56:04.566488 5764 consensus_queue.cc:1035] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 14b5d72bacc54d1d93b06b7135e58a13 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5deb73e487847b5b25c85285c1038b2" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 39335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:56:04.569233 5772 mvcc.cc:205] Tried to move back new op lower bound from 7316544571659452416 to 7316544571590823936. Current Snapshot: MvccSnapshot[applied={T|T < 7316544571659452416}]
I20260809 08:56:04.572002 5016 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55314:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:56:04.572150 5016 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55314:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:56:04.572943 5016 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 04800fcafe48437ea2c4e20605cbc460: Sending DeleteTablet for 3 replicas of tablet 159a541cd3ca4b9ea80cc9d280f85ee3
I20260809 08:56:04.573402 5204 tablet_service.cc:1514] Processing DeleteTablet for tablet 159a541cd3ca4b9ea80cc9d280f85ee3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:56:04 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:33150
I20260809 08:56:04.573540 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 5110
I20260809 08:56:04.573640 5622 tablet_service.cc:1514] Processing DeleteTablet for tablet 159a541cd3ca4b9ea80cc9d280f85ee3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:56:04 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:42276
I20260809 08:56:04.573838 5407 tablet_service.cc:1514] Processing DeleteTablet for tablet 159a541cd3ca4b9ea80cc9d280f85ee3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:56:04 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:49728
I20260809 08:56:04.584089 5784 tablet_replica.cc:330] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 39657962b3284ed7aeb1b5299a74fae4: stopping tablet replica
I20260809 08:56:04.584321 5784 raft_consensus.cc:2229] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 39657962b3284ed7aeb1b5299a74fae4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:56:04.584527 5784 raft_consensus.cc:2258] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 39657962b3284ed7aeb1b5299a74fae4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:56:04.585506 5784 ts_tablet_manager.cc:1905] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 39657962b3284ed7aeb1b5299a74fae4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:56:04.587673 5784 ts_tablet_manager.cc:1918] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 39657962b3284ed7aeb1b5299a74fae4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:56:04.587777 5784 log.cc:1196] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 39657962b3284ed7aeb1b5299a74fae4: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-1/wal/wals/159a541cd3ca4b9ea80cc9d280f85ee3
I20260809 08:56:04.588255 5784 ts_tablet_manager.cc:1939] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 39657962b3284ed7aeb1b5299a74fae4: Deleting consensus metadata
I20260809 08:56:04.589334 4995 catalog_manager.cc:4841] TS 39657962b3284ed7aeb1b5299a74fae4 (127.8.154.2:38653): tablet 159a541cd3ca4b9ea80cc9d280f85ee3 (table test-table [id=0620fc2d4fa247fcad7360a9d3968866]) successfully deleted
I20260809 08:56:04.589395 5782 tablet_replica.cc:330] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 14b5d72bacc54d1d93b06b7135e58a13: stopping tablet replica
I20260809 08:56:04.589650 5782 raft_consensus.cc:2229] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 14b5d72bacc54d1d93b06b7135e58a13 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:56:04.589818 5782 raft_consensus.cc:2258] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 14b5d72bacc54d1d93b06b7135e58a13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:56:04.590390 5782 ts_tablet_manager.cc:1905] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 14b5d72bacc54d1d93b06b7135e58a13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:56:04.592247 5782 ts_tablet_manager.cc:1918] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 14b5d72bacc54d1d93b06b7135e58a13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:56:04.592336 5782 log.cc:1196] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 14b5d72bacc54d1d93b06b7135e58a13: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265519942966-8808-0/minicluster-data/ts-2/wal/wals/159a541cd3ca4b9ea80cc9d280f85ee3
I20260809 08:56:04.592675 5782 ts_tablet_manager.cc:1939] T 159a541cd3ca4b9ea80cc9d280f85ee3 P 14b5d72bacc54d1d93b06b7135e58a13: Deleting consensus metadata
I20260809 08:56:04.593179 4997 catalog_manager.cc:4841] TS 14b5d72bacc54d1d93b06b7135e58a13 (127.8.154.3:46765): tablet 159a541cd3ca4b9ea80cc9d280f85ee3 (table test-table [id=0620fc2d4fa247fcad7360a9d3968866]) successfully deleted
W20260809 08:56:04.594764 4995 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.154.1:39335 (error 108)
I20260809 08:56:04.595357 5743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:04.595209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:39335 (local address 127.0.0.1:33164)
0809 08:56:04.595273 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:04.595316 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.1:39335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 08:56:04.595465 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 5309
W20260809 08:56:04.620028 4995 catalog_manager.cc:4584] TS d5deb73e487847b5b25c85285c1038b2 (127.8.154.1:39335): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 159a541cd3ca4b9ea80cc9d280f85ee3: Network error: Client connection negotiation failed: client connection to 127.8.154.1:39335: connect: Connection refused (error 111)
I20260809 08:56:04.626840 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 5521
I20260809 08:56:04.639631 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 4969
2026-08-09T08:56:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2288 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T08:56:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:56:04Z Disabled control of system clock
I20260809 08:56:04.673038 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:44463
--webserver_interface=127.8.154.62
--webserver_port=0
--builtin_ntp_servers=127.8.154.20:37343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:44463
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 08:56:04.760603 5790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:04.760821 5790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:04.760841 5790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:04.762017 5790 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:56:04.762063 5790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:04.762075 5790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:04.762094 5790 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:56:04.762144 5790 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:56:04.763494 5790 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:37343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:44463
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:44463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:04.763746 5790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:04.764052 5790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:04.769008 5800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:04.769063 5798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:04.769035 5802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:04.769219 5790 server_base.cc:1034] running on GCE node
I20260809 08:56:04.769529 5790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:04.769835 5790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:04.771001 5790 hybrid_clock.cc:648] HybridClock initialized: now 1786265764770987 us; error 44 us; skew 500 ppm
I20260809 08:56:04.772307 5790 webserver.cc:460] Webserver started at http://127.8.154.62:34779/ using document root <none> and password file <none>
I20260809 08:56:04.772567 5790 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:04.772637 5790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:04.772783 5790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:04.773887 5790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "1029fb4636da4549a00f979006223372"
format_stamp: "Formatted at 2026-08-09 08:56:04 on dist-test-slave-t3q3"
server_key: "52c81f734146b646d0965b83c6eca5f1"
server_key_iv: "db24427d610afd7a7ef793d0c80baee6"
server_key_version: "encryptionkey@0"
I20260809 08:56:04.774252 5790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "1029fb4636da4549a00f979006223372"
format_stamp: "Formatted at 2026-08-09 08:56:04 on dist-test-slave-t3q3"
server_key: "52c81f734146b646d0965b83c6eca5f1"
server_key_iv: "db24427d610afd7a7ef793d0c80baee6"
server_key_version: "encryptionkey@0"
I20260809 08:56:04.775575 5790 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:56:04.776391 5813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:04.776577 5790 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:56:04.776680 5790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "1029fb4636da4549a00f979006223372"
format_stamp: "Formatted at 2026-08-09 08:56:04 on dist-test-slave-t3q3"
server_key: "52c81f734146b646d0965b83c6eca5f1"
server_key_iv: "db24427d610afd7a7ef793d0c80baee6"
server_key_version: "encryptionkey@0"
I20260809 08:56:04.776752 5790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:04.803862 5790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:04.804284 5790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:04.804440 5790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:04.808591 5790 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:44463
I20260809 08:56:04.808641 5889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:44463 every 8 connection(s)
I20260809 08:56:04.809033 5790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:56:04.810175 5890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:04.812841 5890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1029fb4636da4549a00f979006223372: Bootstrap starting.
I20260809 08:56:04.813515 5890 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1029fb4636da4549a00f979006223372: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:04.813851 5890 log.cc:824] T 00000000000000000000000000000000 P 1029fb4636da4549a00f979006223372: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:04.814620 5890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1029fb4636da4549a00f979006223372: No bootstrap required, opened a new log
I20260809 08:56:04.816366 5890 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1029fb4636da4549a00f979006223372 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1029fb4636da4549a00f979006223372" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 44463 } }
I20260809 08:56:04.816538 5890 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1029fb4636da4549a00f979006223372 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:04.816581 5890 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1029fb4636da4549a00f979006223372 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1029fb4636da4549a00f979006223372, State: Initialized, Role: FOLLOWER
I20260809 08:56:04.816747 5890 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1029fb4636da4549a00f979006223372 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1029fb4636da4549a00f979006223372" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 44463 } }
I20260809 08:56:04.816833 5890 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1029fb4636da4549a00f979006223372 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:56:04.816879 5890 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1029fb4636da4549a00f979006223372 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:56:04.816931 5890 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1029fb4636da4549a00f979006223372 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:04.817510 5890 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1029fb4636da4549a00f979006223372 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1029fb4636da4549a00f979006223372" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 44463 } }
I20260809 08:56:04.817636 5890 leader_election.cc:302] T 00000000000000000000000000000000 P 1029fb4636da4549a00f979006223372 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1029fb4636da4549a00f979006223372; no voters:
I20260809 08:56:04.817828 5890 leader_election.cc:288] T 00000000000000000000000000000000 P 1029fb4636da4549a00f979006223372 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:56:04.818029 5895 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1029fb4636da4549a00f979006223372 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:56:04.818110 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 5790
I20260809 08:56:04.818216 5890 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1029fb4636da4549a00f979006223372 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:56:04.818256 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:56:04.818245 5895 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1029fb4636da4549a00f979006223372 [term 1 LEADER]: Becoming Leader. State: Replica: 1029fb4636da4549a00f979006223372, State: Running, Role: LEADER
I20260809 08:56:04.818358 5895 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1029fb4636da4549a00f979006223372 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1029fb4636da4549a00f979006223372" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 44463 } }
I20260809 08:56:04.818492 8808 external_mini_cluster.cc:1428] Setting key 78e235596b6c9c6cfabc71a9ecc68fdb
I20260809 08:56:04.818742 5895 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1029fb4636da4549a00f979006223372 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1029fb4636da4549a00f979006223372. Latest consensus state: current_term: 1 leader_uuid: "1029fb4636da4549a00f979006223372" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1029fb4636da4549a00f979006223372" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 44463 } } }
I20260809 08:56:04.818832 5895 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1029fb4636da4549a00f979006223372 [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:04.818976 5896 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1029fb4636da4549a00f979006223372 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1029fb4636da4549a00f979006223372" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1029fb4636da4549a00f979006223372" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 44463 } } }
I20260809 08:56:04.819074 5909 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:56:04.819126 5896 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1029fb4636da4549a00f979006223372 [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:04.819633 5909 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:56:04.821393 5909 catalog_manager.cc:1347] Generated new cluster ID: 2fdb6377f9fa4bec9b5fbdab880b9cc8
I20260809 08:56:04.821451 5909 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:56:04.825718 5916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:04.819213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41016 (local address 127.8.154.62:44463)
0809 08:56:04.819440 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:04.819450 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:04.819812 (+ 362us) server_negotiation.cc:407] Connection header received
0809 08:56:04.820355 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:04.820365 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:04.820451 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:04.820610 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:04.820846 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.822268 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.823407 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.824019 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.824201 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:04.824960 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:04.824964 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:04.824964 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:56:04.825185 (+ 221us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:04.825227 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:56:04.825242 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:04.825306 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:04.825408 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 08:56:04.825487 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":94,"threads_started":1}
I20260809 08:56:04.831619 5909 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:56:04.832340 5909 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:56:04.840742 5909 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1029fb4636da4549a00f979006223372: Generated new TSK 0
I20260809 08:56:04.840946 5909 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:56:04.843389 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.1:0
--local_ip_for_outbound_sockets=127.8.154.1
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:44463
--builtin_ntp_servers=127.8.154.20:37343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 08:56:04.929271 5926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:04.929508 5926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:04.929548 5926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:04.930712 5926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:04.930773 5926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:04.930819 5926 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.1
I20260809 08:56:04.932261 5926 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:37343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:44463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.1
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:04.932528 5926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:04.932832 5926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:04.937991 5939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:04.937995 5935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:04.938318 5936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:04.938663 5926 server_base.cc:1034] running on GCE node
I20260809 08:56:04.938885 5926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:04.939155 5926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:04.940313 5926 hybrid_clock.cc:648] HybridClock initialized: now 1786265764940300 us; error 31 us; skew 500 ppm
I20260809 08:56:04.941624 5926 webserver.cc:460] Webserver started at http://127.8.154.1:35359/ using document root <none> and password file <none>
I20260809 08:56:04.941884 5926 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:04.941967 5926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:04.942121 5926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:04.943368 5926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-0/data/instance:
uuid: "94381e1c6bc14ce781d3efcd6b4e009f"
format_stamp: "Formatted at 2026-08-09 08:56:04 on dist-test-slave-t3q3"
server_key: "f34d2c2134df62e834b0d6ccf5ca9dde"
server_key_iv: "25f614d1c39fa0a4aec61f7cd67545a6"
server_key_version: "encryptionkey@0"
I20260809 08:56:04.943749 5926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance:
uuid: "94381e1c6bc14ce781d3efcd6b4e009f"
format_stamp: "Formatted at 2026-08-09 08:56:04 on dist-test-slave-t3q3"
server_key: "f34d2c2134df62e834b0d6ccf5ca9dde"
server_key_iv: "25f614d1c39fa0a4aec61f7cd67545a6"
server_key_version: "encryptionkey@0"
I20260809 08:56:04.945189 5926 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:56:04.945971 5947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:04.946163 5926 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:56:04.946271 5926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-0/wal
uuid: "94381e1c6bc14ce781d3efcd6b4e009f"
format_stamp: "Formatted at 2026-08-09 08:56:04 on dist-test-slave-t3q3"
server_key: "f34d2c2134df62e834b0d6ccf5ca9dde"
server_key_iv: "25f614d1c39fa0a4aec61f7cd67545a6"
server_key_version: "encryptionkey@0"
I20260809 08:56:04.946343 5926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:04.961112 5926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:04.961411 5926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:04.961521 5926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:04.961736 5926 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:56:04.962028 5926 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:56:04.962057 5926 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:04.962080 5926 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:56:04.962092 5926 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:04.968003 5926 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.1:36623
I20260809 08:56:04.968221 6126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.1:36623 every 8 connection(s)
I20260809 08:56:04.968520 5926 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
I20260809 08:56:04.969466 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 5926
I20260809 08:56:04.969594 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance
I20260809 08:56:04.969815 8808 external_mini_cluster.cc:1428] Setting key d967060b1ef548c21e9afce6dfe0b7f4
I20260809 08:56:04.970892 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.2:0
--local_ip_for_outbound_sockets=127.8.154.2
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:44463
--builtin_ntp_servers=127.8.154.20:37343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 08:56:04.974691 5916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:04.969268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:57809 (local address 127.8.154.62:44463)
0809 08:56:04.969320 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:04.969323 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:04.969572 (+ 249us) server_negotiation.cc:407] Connection header received
0809 08:56:04.970293 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:04.970295 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:04.970306 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:04.970372 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:04.971234 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.971990 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.973116 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.973437 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.973519 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:04.974387 (+ 868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:04.974389 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:04.974389 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:56:04.974394 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:04.974411 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:56:04.974420 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:04.974479 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:04.974576 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 08:56:04.974610 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:56:04.974911 6129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:04.969237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:44463 (local address 127.8.154.1:57809)
0809 08:56:04.969432 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:04.969476 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:04.970183 (+ 707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:04.970453 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:04.970457 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:04.970467 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:56:04.971144 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:04.971150 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.972134 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.972135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:04.973026 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:04.973029 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:04.973524 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:04.973524 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:04.973638 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:04.974222 (+ 584us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:56:04.974222 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:56:04.974252 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:56:04.974275 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:56:04.974481 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:56:04.974488 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:56:04.974492 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:56:04.974579 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 08:56:04.974685 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":95,"threads_started":1}
I20260809 08:56:04.975463 6127 heartbeater.cc:347] Connected to a master server at 127.8.154.62:44463
I20260809 08:56:04.975565 6127 heartbeater.cc:464] Registering TS with master...
I20260809 08:56:04.975749 6127 heartbeater.cc:511] Master 127.8.154.62:44463 requested a full tablet report, sending...
I20260809 08:56:04.976294 5833 ts_manager.cc:194] Registered new tserver with Master: 94381e1c6bc14ce781d3efcd6b4e009f (127.8.154.1:36623)
I20260809 08:56:04.976912 5833 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.154.1:57809
W20260809 08:56:05.057150 6130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:05.057387 6130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:05.057423 6130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:05.058549 6130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:05.058616 6130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:05.058657 6130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.2
I20260809 08:56:05.060005 6130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:37343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:44463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.2
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:05.060302 6130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:05.060611 6130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:05.065879 6143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:05.065886 6139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:05.065901 6140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:05.066059 6130 server_base.cc:1034] running on GCE node
I20260809 08:56:05.066237 6130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:05.066478 6130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:05.067624 6130 hybrid_clock.cc:648] HybridClock initialized: now 1786265765067607 us; error 33 us; skew 500 ppm
I20260809 08:56:05.068977 6130 webserver.cc:460] Webserver started at http://127.8.154.2:40745/ using document root <none> and password file <none>
I20260809 08:56:05.069238 6130 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:05.069286 6130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:05.069458 6130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:05.070509 6130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-1/data/instance:
uuid: "661d3a8183eb4e10ac2b42580dbedb23"
format_stamp: "Formatted at 2026-08-09 08:56:05 on dist-test-slave-t3q3"
server_key: "8c0bc7cb0331bcb7391d96f42d078b0d"
server_key_iv: "2260db1b570591fcbbc3fae382df5d1c"
server_key_version: "encryptionkey@0"
I20260809 08:56:05.070920 6130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance:
uuid: "661d3a8183eb4e10ac2b42580dbedb23"
format_stamp: "Formatted at 2026-08-09 08:56:05 on dist-test-slave-t3q3"
server_key: "8c0bc7cb0331bcb7391d96f42d078b0d"
server_key_iv: "2260db1b570591fcbbc3fae382df5d1c"
server_key_version: "encryptionkey@0"
I20260809 08:56:05.072708 6130 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 08:56:05.073495 6159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:05.073683 6130 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:56:05.073773 6130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-1/wal
uuid: "661d3a8183eb4e10ac2b42580dbedb23"
format_stamp: "Formatted at 2026-08-09 08:56:05 on dist-test-slave-t3q3"
server_key: "8c0bc7cb0331bcb7391d96f42d078b0d"
server_key_iv: "2260db1b570591fcbbc3fae382df5d1c"
server_key_version: "encryptionkey@0"
I20260809 08:56:05.073846 6130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:05.089941 6130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:05.090281 6130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:05.090476 6130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:05.090754 6130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:56:05.091102 6130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:56:05.091156 6130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:05.091197 6130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:56:05.091224 6130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:05.097261 6130 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.2:38729
I20260809 08:56:05.097294 6331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.2:38729 every 8 connection(s)
I20260809 08:56:05.097661 6130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
I20260809 08:56:05.103149 5916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:05.098489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:40875 (local address 127.8.154.62:44463)
0809 08:56:05.098550 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:05.098552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:05.098800 (+ 248us) server_negotiation.cc:407] Connection header received
0809 08:56:05.099321 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:05.099322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:05.099331 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:05.099395 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:05.100179 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:05.100674 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:05.101706 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:05.101893 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:05.101946 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:05.102918 (+ 972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:05.102919 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:05.102919 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:56:05.102924 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:05.102934 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:56:05.102937 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:05.102975 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:05.103056 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 08:56:05.103077 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 08:56:05.103369 6335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:05.098451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:44463 (local address 127.8.154.2:40875)
0809 08:56:05.098666 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:05.098709 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:05.099232 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:05.099477 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:05.099480 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:05.099489 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:56:05.100002 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:05.100006 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:05.100787 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:05.100788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:05.101626 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:05.101629 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:05.101992 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:05.101993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:05.102123 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:05.102791 (+ 668us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:56:05.102791 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:56:05.102823 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:56:05.102848 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:56:05.102989 (+ 141us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:56:05.102994 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:56:05.102997 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:56:05.103055 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 08:56:05.103146 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":109,"threads_started":1}
I20260809 08:56:05.103906 6333 heartbeater.cc:347] Connected to a master server at 127.8.154.62:44463
I20260809 08:56:05.104014 6333 heartbeater.cc:464] Registering TS with master...
I20260809 08:56:05.104230 6333 heartbeater.cc:511] Master 127.8.154.62:44463 requested a full tablet report, sending...
I20260809 08:56:05.104626 5833 ts_manager.cc:194] Registered new tserver with Master: 661d3a8183eb4e10ac2b42580dbedb23 (127.8.154.2:38729)
I20260809 08:56:05.104997 5833 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.154.2:40875
I20260809 08:56:05.107528 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 6130
I20260809 08:56:05.107650 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance
I20260809 08:56:05.107863 8808 external_mini_cluster.cc:1428] Setting key a621ede1291b969d1337bcde072da127
I20260809 08:56:05.108894 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.3:0
--local_ip_for_outbound_sockets=127.8.154.3
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:44463
--builtin_ntp_servers=127.8.154.20:37343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 08:56:05.196147 6337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:05.196393 6337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:05.196431 6337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:05.197569 6337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:05.197638 6337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:05.197678 6337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.3
I20260809 08:56:05.199035 6337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:37343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:44463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.3
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:05.199302 6337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:05.199612 6337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:05.204881 6352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:05.204916 6346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:05.204871 6349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:05.205217 6337 server_base.cc:1034] running on GCE node
I20260809 08:56:05.205466 6337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:05.205686 6337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:05.206830 6337 hybrid_clock.cc:648] HybridClock initialized: now 1786265765206819 us; error 33 us; skew 500 ppm
I20260809 08:56:05.208382 6337 webserver.cc:460] Webserver started at http://127.8.154.3:40857/ using document root <none> and password file <none>
I20260809 08:56:05.208654 6337 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:05.208714 6337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:05.208895 6337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:05.209972 6337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-2/data/instance:
uuid: "f85265cb4a0848ad98fda72d9a20b1f2"
format_stamp: "Formatted at 2026-08-09 08:56:05 on dist-test-slave-t3q3"
server_key: "dc4c7a6b7c71bbee33c264870a6d8322"
server_key_iv: "3428f4207f0a6462c28008343b1b9ab5"
server_key_version: "encryptionkey@0"
I20260809 08:56:05.210330 6337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance:
uuid: "f85265cb4a0848ad98fda72d9a20b1f2"
format_stamp: "Formatted at 2026-08-09 08:56:05 on dist-test-slave-t3q3"
server_key: "dc4c7a6b7c71bbee33c264870a6d8322"
server_key_iv: "3428f4207f0a6462c28008343b1b9ab5"
server_key_version: "encryptionkey@0"
I20260809 08:56:05.211637 6337 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:56:05.212414 6365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:05.212584 6337 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:05.212687 6337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-2/wal
uuid: "f85265cb4a0848ad98fda72d9a20b1f2"
format_stamp: "Formatted at 2026-08-09 08:56:05 on dist-test-slave-t3q3"
server_key: "dc4c7a6b7c71bbee33c264870a6d8322"
server_key_iv: "3428f4207f0a6462c28008343b1b9ab5"
server_key_version: "encryptionkey@0"
I20260809 08:56:05.212757 6337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:05.223778 6337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:05.224150 6337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:05.224306 6337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:05.224542 6337 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:56:05.224884 6337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:56:05.224936 6337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:05.224977 6337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:56:05.225003 6337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:05.230952 6337 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.3:45981
I20260809 08:56:05.231014 6556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.3:45981 every 8 connection(s)
I20260809 08:56:05.231340 6337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
I20260809 08:56:05.234586 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 6337
I20260809 08:56:05.234715 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance
I20260809 08:56:05.234938 8808 external_mini_cluster.cc:1428] Setting key f6665041565b91c419e84ead2047a908
I20260809 08:56:05.236807 5916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:05.232115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:40877 (local address 127.8.154.62:44463)
0809 08:56:05.232165 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:05.232167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:05.232435 (+ 268us) server_negotiation.cc:407] Connection header received
0809 08:56:05.232990 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:05.232991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:05.233004 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:05.233054 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:05.233760 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:05.234266 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:05.235388 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:05.235591 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:05.235664 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:05.236494 (+ 830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:05.236496 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:05.236496 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:56:05.236520 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:05.236535 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:56:05.236539 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:05.236602 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:05.236692 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 08:56:05.236724 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 08:56:05.237160 6559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:05.232100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:44463 (local address 127.8.154.3:40877)
0809 08:56:05.232314 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:05.232347 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:05.232866 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:05.233124 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:05.233128 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:05.233142 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:56:05.233656 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:05.233660 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:05.234359 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:05.234360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:05.235298 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:05.235301 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:05.235685 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:05.235686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:05.235814 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:05.236355 (+ 541us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:56:05.236355 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:56:05.236385 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:56:05.236408 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:56:05.236604 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:56:05.236611 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:56:05.236615 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:56:05.236696 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 08:56:05.236798 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":104,"threads_started":1}
I20260809 08:56:05.237663 6557 heartbeater.cc:347] Connected to a master server at 127.8.154.62:44463
I20260809 08:56:05.237751 6557 heartbeater.cc:464] Registering TS with master...
I20260809 08:56:05.237895 6557 heartbeater.cc:511] Master 127.8.154.62:44463 requested a full tablet report, sending...
I20260809 08:56:05.238230 5833 ts_manager.cc:194] Registered new tserver with Master: f85265cb4a0848ad98fda72d9a20b1f2 (127.8.154.3:45981)
I20260809 08:56:05.238606 5833 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.154.3:40877
I20260809 08:56:05.246760 8808 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:56:05.251730 5916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:05.248429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41024 (local address 127.8.154.62:44463)
0809 08:56:05.248517 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:05.248519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:05.248620 (+ 101us) server_negotiation.cc:407] Connection header received
0809 08:56:05.248715 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:05.248716 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:05.248730 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:05.248796 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:05.248976 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:05.249460 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:05.250270 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:05.250464 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:05.250556 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:05.251398 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:05.251400 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:05.251400 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:56:05.251405 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:05.251420 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:56:05.251425 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:05.251531 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:05.251629 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 08:56:05.251641 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 08:56:05.253512 5833 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41024:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:56:05.255002 5833 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:56:05.261624 6569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:05.257314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:38729 (local address 127.0.0.1:38486)
0809 08:56:05.257564 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:05.257609 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:05.257683 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:05.258219 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:05.258219 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:05.258222 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:56:05.258254 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:05.258255 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:05.259574 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:05.259575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:05.260519 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:05.260522 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:05.261379 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:05.261380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:05.261443 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:05.261455 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 08:56:05.261521 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 08:56:05.261562 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"mutex_wait_us":1,"thread_start_us":118,"threads_started":1}
I20260809 08:56:05.261677 6577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:05.257800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38486 (local address 127.8.154.2:38729)
0809 08:56:05.258039 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:05.258044 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:05.258053 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:56:05.258092 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:05.258094 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:05.258109 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:05.258334 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:56:05.258401 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:05.259513 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:05.260610 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:05.261289 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:05.261368 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:05.261464 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:05.261536 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 08:56:05.261572 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":113,"threads_started":1}
I20260809 08:56:05.261972 6568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:05.257317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:45981 (local address 127.0.0.1:44070)
0809 08:56:05.257773 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:05.257781 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:05.257820 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:05.258074 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:05.258078 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:05.258092 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:56:05.258165 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:05.258167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:05.259529 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:05.259530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:05.260775 (+ 1245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:05.260778 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:05.261717 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:05.261717 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:05.261790 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:05.261795 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:56:05.261843 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 08:56:05.261889 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"mutex_wait_us":20,"thread_start_us":74,"threads_started":1}
I20260809 08:56:05.262610 6236 tablet_service.cc:1467] Processing CreateTablet for tablet ca552818b66343f492d11ac1c4c28875 (DEFAULT_TABLE table=test-table [id=5889eae5ecfd4bef89c79ad73bb0e3c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:56:05.263072 6236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca552818b66343f492d11ac1c4c28875. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:05.263182 6572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:05.257317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:36623 (local address 127.0.0.1:43958)
0809 08:56:05.257874 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:05.257882 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:05.257936 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:05.258272 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:05.258273 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:05.258275 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:56:05.258329 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:05.258331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:05.259294 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:05.259295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:05.260775 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:05.260778 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:05.262908 (+ 2130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:05.262909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:05.262990 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:05.262996 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 08:56:05.263063 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 08:56:05.263112 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":121,"threads_started":1}
I20260809 08:56:05.263217 6574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:05.257597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44070 (local address 127.8.154.3:45981)
0809 08:56:05.257766 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:05.257772 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:05.257820 (+ 48us) server_negotiation.cc:407] Connection header received
0809 08:56:05.257875 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:05.257877 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:05.257895 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:05.257982 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:56:05.258254 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:05.259435 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:05.260878 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:05.261624 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:05.262994 (+ 1370us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:05.263058 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:05.263086 (+ 28us) server_negotiation.cc:299] Negotiation successful
0809 08:56:05.263118 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":101,"threads_started":1}
I20260809 08:56:05.263424 6573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:05.257443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43958 (local address 127.8.154.1:36623)
0809 08:56:05.258067 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:05.258073 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:05.258084 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:56:05.258136 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:05.258138 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:05.258154 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:05.258405 (+ 251us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:56:05.258454 (+ 49us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:05.259201 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:05.261992 (+ 2791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:05.262803 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:05.262888 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:05.262974 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:05.263295 (+ 321us) server_negotiation.cc:299] Negotiation successful
0809 08:56:05.263344 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":101,"threads_started":1}
I20260809 08:56:05.263877 6445 tablet_service.cc:1467] Processing CreateTablet for tablet ca552818b66343f492d11ac1c4c28875 (DEFAULT_TABLE table=test-table [id=5889eae5ecfd4bef89c79ad73bb0e3c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:56:05.264232 6445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca552818b66343f492d11ac1c4c28875. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:05.264794 6019 tablet_service.cc:1467] Processing CreateTablet for tablet ca552818b66343f492d11ac1c4c28875 (DEFAULT_TABLE table=test-table [id=5889eae5ecfd4bef89c79ad73bb0e3c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:56:05.265131 6019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca552818b66343f492d11ac1c4c28875. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:05.266525 6579 tablet_bootstrap.cc:492] T ca552818b66343f492d11ac1c4c28875 P 661d3a8183eb4e10ac2b42580dbedb23: Bootstrap starting.
I20260809 08:56:05.267354 6579 tablet_bootstrap.cc:654] T ca552818b66343f492d11ac1c4c28875 P 661d3a8183eb4e10ac2b42580dbedb23: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:05.267688 6581 tablet_bootstrap.cc:492] T ca552818b66343f492d11ac1c4c28875 P f85265cb4a0848ad98fda72d9a20b1f2: Bootstrap starting.
I20260809 08:56:05.267747 6579 log.cc:824] T ca552818b66343f492d11ac1c4c28875 P 661d3a8183eb4e10ac2b42580dbedb23: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:05.268532 6581 tablet_bootstrap.cc:654] T ca552818b66343f492d11ac1c4c28875 P f85265cb4a0848ad98fda72d9a20b1f2: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:05.268527 6585 tablet_bootstrap.cc:492] T ca552818b66343f492d11ac1c4c28875 P 94381e1c6bc14ce781d3efcd6b4e009f: Bootstrap starting.
I20260809 08:56:05.268671 6579 tablet_bootstrap.cc:492] T ca552818b66343f492d11ac1c4c28875 P 661d3a8183eb4e10ac2b42580dbedb23: No bootstrap required, opened a new log
I20260809 08:56:05.268904 6579 ts_tablet_manager.cc:1397] T ca552818b66343f492d11ac1c4c28875 P 661d3a8183eb4e10ac2b42580dbedb23: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:56:05.269187 6585 tablet_bootstrap.cc:654] T ca552818b66343f492d11ac1c4c28875 P 94381e1c6bc14ce781d3efcd6b4e009f: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:05.269228 6581 log.cc:824] T ca552818b66343f492d11ac1c4c28875 P f85265cb4a0848ad98fda72d9a20b1f2: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:05.269510 6585 log.cc:824] T ca552818b66343f492d11ac1c4c28875 P 94381e1c6bc14ce781d3efcd6b4e009f: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:05.270157 6581 tablet_bootstrap.cc:492] T ca552818b66343f492d11ac1c4c28875 P f85265cb4a0848ad98fda72d9a20b1f2: No bootstrap required, opened a new log
I20260809 08:56:05.270241 6581 ts_tablet_manager.cc:1397] T ca552818b66343f492d11ac1c4c28875 P f85265cb4a0848ad98fda72d9a20b1f2: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 08:56:05.272008 6579 raft_consensus.cc:348] T ca552818b66343f492d11ac1c4c28875 P 661d3a8183eb4e10ac2b42580dbedb23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661d3a8183eb4e10ac2b42580dbedb23" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38729 } } peers { permanent_uuid: "94381e1c6bc14ce781d3efcd6b4e009f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 36623 } } peers { permanent_uuid: "f85265cb4a0848ad98fda72d9a20b1f2" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45981 } }
I20260809 08:56:05.272233 6579 raft_consensus.cc:374] T ca552818b66343f492d11ac1c4c28875 P 661d3a8183eb4e10ac2b42580dbedb23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:05.272289 6579 raft_consensus.cc:729] T ca552818b66343f492d11ac1c4c28875 P 661d3a8183eb4e10ac2b42580dbedb23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 661d3a8183eb4e10ac2b42580dbedb23, State: Initialized, Role: FOLLOWER
I20260809 08:56:05.272418 6579 consensus_queue.cc:260] T ca552818b66343f492d11ac1c4c28875 P 661d3a8183eb4e10ac2b42580dbedb23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661d3a8183eb4e10ac2b42580dbedb23" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38729 } } peers { permanent_uuid: "94381e1c6bc14ce781d3efcd6b4e009f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 36623 } } peers { permanent_uuid: "f85265cb4a0848ad98fda72d9a20b1f2" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45981 } }
I20260809 08:56:05.272714 6579 ts_tablet_manager.cc:1428] T ca552818b66343f492d11ac1c4c28875 P 661d3a8183eb4e10ac2b42580dbedb23: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 08:56:05.272784 6585 tablet_bootstrap.cc:492] T ca552818b66343f492d11ac1c4c28875 P 94381e1c6bc14ce781d3efcd6b4e009f: No bootstrap required, opened a new log
I20260809 08:56:05.272859 6333 heartbeater.cc:503] Master 127.8.154.62:44463 was elected leader, sending a full tablet report...
I20260809 08:56:05.272873 6585 ts_tablet_manager.cc:1397] T ca552818b66343f492d11ac1c4c28875 P 94381e1c6bc14ce781d3efcd6b4e009f: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260809 08:56:05.273341 6581 raft_consensus.cc:348] T ca552818b66343f492d11ac1c4c28875 P f85265cb4a0848ad98fda72d9a20b1f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661d3a8183eb4e10ac2b42580dbedb23" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38729 } } peers { permanent_uuid: "94381e1c6bc14ce781d3efcd6b4e009f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 36623 } } peers { permanent_uuid: "f85265cb4a0848ad98fda72d9a20b1f2" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45981 } }
I20260809 08:56:05.273521 6581 raft_consensus.cc:374] T ca552818b66343f492d11ac1c4c28875 P f85265cb4a0848ad98fda72d9a20b1f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:05.273562 6581 raft_consensus.cc:729] T ca552818b66343f492d11ac1c4c28875 P f85265cb4a0848ad98fda72d9a20b1f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f85265cb4a0848ad98fda72d9a20b1f2, State: Initialized, Role: FOLLOWER
I20260809 08:56:05.273710 6581 consensus_queue.cc:260] T ca552818b66343f492d11ac1c4c28875 P f85265cb4a0848ad98fda72d9a20b1f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661d3a8183eb4e10ac2b42580dbedb23" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38729 } } peers { permanent_uuid: "94381e1c6bc14ce781d3efcd6b4e009f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 36623 } } peers { permanent_uuid: "f85265cb4a0848ad98fda72d9a20b1f2" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45981 } }
I20260809 08:56:05.274040 6581 ts_tablet_manager.cc:1428] T ca552818b66343f492d11ac1c4c28875 P f85265cb4a0848ad98fda72d9a20b1f2: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260809 08:56:05.274086 6557 heartbeater.cc:503] Master 127.8.154.62:44463 was elected leader, sending a full tablet report...
I20260809 08:56:05.275409 6585 raft_consensus.cc:348] T ca552818b66343f492d11ac1c4c28875 P 94381e1c6bc14ce781d3efcd6b4e009f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661d3a8183eb4e10ac2b42580dbedb23" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38729 } } peers { permanent_uuid: "94381e1c6bc14ce781d3efcd6b4e009f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 36623 } } peers { permanent_uuid: "f85265cb4a0848ad98fda72d9a20b1f2" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45981 } }
I20260809 08:56:05.275570 6585 raft_consensus.cc:374] T ca552818b66343f492d11ac1c4c28875 P 94381e1c6bc14ce781d3efcd6b4e009f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:05.275624 6585 raft_consensus.cc:729] T ca552818b66343f492d11ac1c4c28875 P 94381e1c6bc14ce781d3efcd6b4e009f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94381e1c6bc14ce781d3efcd6b4e009f, State: Initialized, Role: FOLLOWER
I20260809 08:56:05.275744 6585 consensus_queue.cc:260] T ca552818b66343f492d11ac1c4c28875 P 94381e1c6bc14ce781d3efcd6b4e009f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661d3a8183eb4e10ac2b42580dbedb23" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38729 } } peers { permanent_uuid: "94381e1c6bc14ce781d3efcd6b4e009f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 36623 } } peers { permanent_uuid: "f85265cb4a0848ad98fda72d9a20b1f2" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45981 } }
I20260809 08:56:05.276014 6585 ts_tablet_manager.cc:1428] T ca552818b66343f492d11ac1c4c28875 P 94381e1c6bc14ce781d3efcd6b4e009f: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260809 08:56:05.276168 6127 heartbeater.cc:503] Master 127.8.154.62:44463 was elected leader, sending a full tablet report...
W20260809 08:56:05.348284 6334 tablet.cc:2369] T ca552818b66343f492d11ac1c4c28875 P 661d3a8183eb4e10ac2b42580dbedb23: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:56:05.390437 6592 raft_consensus.cc:482] T ca552818b66343f492d11ac1c4c28875 P 94381e1c6bc14ce781d3efcd6b4e009f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:56:05.390653 6592 raft_consensus.cc:504] T ca552818b66343f492d11ac1c4c28875 P 94381e1c6bc14ce781d3efcd6b4e009f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661d3a8183eb4e10ac2b42580dbedb23" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38729 } } peers { permanent_uuid: "94381e1c6bc14ce781d3efcd6b4e009f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 36623 } } peers { permanent_uuid: "f85265cb4a0848ad98fda72d9a20b1f2" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45981 } }
I20260809 08:56:05.391103 6592 leader_election.cc:288] T ca552818b66343f492d11ac1c4c28875 P 94381e1c6bc14ce781d3efcd6b4e009f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 661d3a8183eb4e10ac2b42580dbedb23 (127.8.154.2:38729), f85265cb4a0848ad98fda72d9a20b1f2 (127.8.154.3:45981)
I20260809 08:56:05.394886 6574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:05.391472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:51997 (local address 127.8.154.3:45981)
0809 08:56:05.391530 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:05.391533 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:05.391736 (+ 203us) server_negotiation.cc:407] Connection header received
0809 08:56:05.391772 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:05.391773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:05.391783 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:05.391845 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:56:05.392040 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:05.392731 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:05.394017 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:05.394542 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:05.394617 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:05.394626 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:05.394771 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 08:56:05.394812 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 08:56:05.394886 6577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:05.391390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:40523 (local address 127.8.154.2:38729)
0809 08:56:05.391438 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:05.391440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:05.391453 (+ 13us) server_negotiation.cc:407] Connection header received
0809 08:56:05.391487 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:05.391488 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:05.391497 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:05.391559 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:56:05.391758 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:05.392538 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:05.393927 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:05.394511 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:05.394595 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:05.394607 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:05.394780 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 08:56:05.394822 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 08:56:05.394891 6129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:05.391300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:38729 (local address 127.8.154.1:40523)
0809 08:56:05.391359 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:05.391373 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:05.391429 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:05.391585 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:05.391586 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:05.391589 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:56:05.391672 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:05.391675 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:05.392617 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:05.392617 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:05.393827 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:05.393830 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:05.394607 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:05.394608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:05.394682 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:05.394687 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:56:05.394750 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 08:56:05.394796 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 08:56:05.394896 6593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:05.391434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:45981 (local address 127.8.154.1:51997)
0809 08:56:05.391640 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:05.391652 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:05.391719 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:05.391890 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:05.391891 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:05.391893 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:56:05.391960 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:05.391962 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:05.392827 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:05.392828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:05.393945 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:05.393946 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:05.394628 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:05.394628 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:05.394700 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:05.394703 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:56:05.394755 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 08:56:05.394791 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":3,"thread_start_us":123,"threads_started":1}
I20260809 08:56:05.395399 6269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca552818b66343f492d11ac1c4c28875" candidate_uuid: "94381e1c6bc14ce781d3efcd6b4e009f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "661d3a8183eb4e10ac2b42580dbedb23" is_pre_election: true
I20260809 08:56:05.395524 6490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca552818b66343f492d11ac1c4c28875" candidate_uuid: "94381e1c6bc14ce781d3efcd6b4e009f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f85265cb4a0848ad98fda72d9a20b1f2" is_pre_election: true
I20260809 08:56:05.395586 6269 raft_consensus.cc:2454] T ca552818b66343f492d11ac1c4c28875 P 661d3a8183eb4e10ac2b42580dbedb23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94381e1c6bc14ce781d3efcd6b4e009f in term 0.
I20260809 08:56:05.395714 6490 raft_consensus.cc:2454] T ca552818b66343f492d11ac1c4c28875 P f85265cb4a0848ad98fda72d9a20b1f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94381e1c6bc14ce781d3efcd6b4e009f in term 0.
I20260809 08:56:05.395864 5951 leader_election.cc:302] T ca552818b66343f492d11ac1c4c28875 P 94381e1c6bc14ce781d3efcd6b4e009f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 661d3a8183eb4e10ac2b42580dbedb23, 94381e1c6bc14ce781d3efcd6b4e009f; no voters:
I20260809 08:56:05.395998 6592 raft_consensus.cc:2781] T ca552818b66343f492d11ac1c4c28875 P 94381e1c6bc14ce781d3efcd6b4e009f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:56:05.396119 6592 raft_consensus.cc:482] T ca552818b66343f492d11ac1c4c28875 P 94381e1c6bc14ce781d3efcd6b4e009f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:56:05.396166 6592 raft_consensus.cc:3037] T ca552818b66343f492d11ac1c4c28875 P 94381e1c6bc14ce781d3efcd6b4e009f [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:05.396924 6592 raft_consensus.cc:504] T ca552818b66343f492d11ac1c4c28875 P 94381e1c6bc14ce781d3efcd6b4e009f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661d3a8183eb4e10ac2b42580dbedb23" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38729 } } peers { permanent_uuid: "94381e1c6bc14ce781d3efcd6b4e009f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 36623 } } peers { permanent_uuid: "f85265cb4a0848ad98fda72d9a20b1f2" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45981 } }
I20260809 08:56:05.397078 6592 leader_election.cc:288] T ca552818b66343f492d11ac1c4c28875 P 94381e1c6bc14ce781d3efcd6b4e009f [CANDIDATE]: Term 1 election: Requested vote from peers 661d3a8183eb4e10ac2b42580dbedb23 (127.8.154.2:38729), f85265cb4a0848ad98fda72d9a20b1f2 (127.8.154.3:45981)
I20260809 08:56:05.397258 6269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca552818b66343f492d11ac1c4c28875" candidate_uuid: "94381e1c6bc14ce781d3efcd6b4e009f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "661d3a8183eb4e10ac2b42580dbedb23"
I20260809 08:56:05.397287 6490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca552818b66343f492d11ac1c4c28875" candidate_uuid: "94381e1c6bc14ce781d3efcd6b4e009f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f85265cb4a0848ad98fda72d9a20b1f2"
I20260809 08:56:05.397359 6490 raft_consensus.cc:3037] T ca552818b66343f492d11ac1c4c28875 P f85265cb4a0848ad98fda72d9a20b1f2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:05.397363 6269 raft_consensus.cc:3037] T ca552818b66343f492d11ac1c4c28875 P 661d3a8183eb4e10ac2b42580dbedb23 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:05.398102 6490 raft_consensus.cc:2454] T ca552818b66343f492d11ac1c4c28875 P f85265cb4a0848ad98fda72d9a20b1f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94381e1c6bc14ce781d3efcd6b4e009f in term 1.
I20260809 08:56:05.398180 6269 raft_consensus.cc:2454] T ca552818b66343f492d11ac1c4c28875 P 661d3a8183eb4e10ac2b42580dbedb23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94381e1c6bc14ce781d3efcd6b4e009f in term 1.
I20260809 08:56:05.398319 5953 leader_election.cc:302] T ca552818b66343f492d11ac1c4c28875 P 94381e1c6bc14ce781d3efcd6b4e009f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94381e1c6bc14ce781d3efcd6b4e009f, f85265cb4a0848ad98fda72d9a20b1f2; no voters:
I20260809 08:56:05.398443 6592 raft_consensus.cc:2781] T ca552818b66343f492d11ac1c4c28875 P 94381e1c6bc14ce781d3efcd6b4e009f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:56:05.398600 6592 raft_consensus.cc:686] T ca552818b66343f492d11ac1c4c28875 P 94381e1c6bc14ce781d3efcd6b4e009f [term 1 LEADER]: Becoming Leader. State: Replica: 94381e1c6bc14ce781d3efcd6b4e009f, State: Running, Role: LEADER
I20260809 08:56:05.398720 6592 consensus_queue.cc:237] T ca552818b66343f492d11ac1c4c28875 P 94381e1c6bc14ce781d3efcd6b4e009f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661d3a8183eb4e10ac2b42580dbedb23" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38729 } } peers { permanent_uuid: "94381e1c6bc14ce781d3efcd6b4e009f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 36623 } } peers { permanent_uuid: "f85265cb4a0848ad98fda72d9a20b1f2" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45981 } }
I20260809 08:56:05.399595 5832 catalog_manager.cc:5495] T ca552818b66343f492d11ac1c4c28875 P 94381e1c6bc14ce781d3efcd6b4e009f reported cstate change: term changed from 0 to 1, leader changed from <none> to 94381e1c6bc14ce781d3efcd6b4e009f (127.8.154.1). New cstate: current_term: 1 leader_uuid: "94381e1c6bc14ce781d3efcd6b4e009f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "661d3a8183eb4e10ac2b42580dbedb23" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "94381e1c6bc14ce781d3efcd6b4e009f" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 36623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f85265cb4a0848ad98fda72d9a20b1f2" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45981 } health_report { overall_health: UNKNOWN } } }
I20260809 08:56:05.406011 6573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:05.403533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43962 (local address 127.8.154.1:36623)
0809 08:56:05.403587 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:05.403589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:05.403597 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:56:05.403683 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:05.403684 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:05.403705 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:05.403792 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:56:05.403979 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:05.404489 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:05.405244 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:05.405448 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:05.405517 (+ 69us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:05.405655 (+ 138us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:56:05.405753 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:56:05.405861 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:05.405893 (+ 32us) server_negotiation.cc:299] Negotiation successful
0809 08:56:05.405904 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 08:56:05.407712 6490 raft_consensus.cc:1261] T ca552818b66343f492d11ac1c4c28875 P f85265cb4a0848ad98fda72d9a20b1f2 [term 1 FOLLOWER]: Refusing update from remote peer 94381e1c6bc14ce781d3efcd6b4e009f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:56:05.407712 6269 raft_consensus.cc:1261] T ca552818b66343f492d11ac1c4c28875 P 661d3a8183eb4e10ac2b42580dbedb23 [term 1 FOLLOWER]: Refusing update from remote peer 94381e1c6bc14ce781d3efcd6b4e009f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:56:05.408046 6594 consensus_queue.cc:1035] T ca552818b66343f492d11ac1c4c28875 P 94381e1c6bc14ce781d3efcd6b4e009f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f85265cb4a0848ad98fda72d9a20b1f2" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:56:05.408191 6592 consensus_queue.cc:1035] T ca552818b66343f492d11ac1c4c28875 P 94381e1c6bc14ce781d3efcd6b4e009f [LEADER]: Connected to new peer: Peer: permanent_uuid: "661d3a8183eb4e10ac2b42580dbedb23" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 38729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:56:05.410765 6599 mvcc.cc:205] Tried to move back new op lower bound from 7316544575107452928 to 7316544575073865728. Current Snapshot: MvccSnapshot[applied={T|T < 7316544575107452928}]
I20260809 08:56:05.412997 5833 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:56:05.413092 5833 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:56:05.413844 5833 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1029fb4636da4549a00f979006223372: Sending DeleteTablet for 3 replicas of tablet ca552818b66343f492d11ac1c4c28875
I20260809 08:56:05.414175 6019 tablet_service.cc:1514] Processing DeleteTablet for tablet ca552818b66343f492d11ac1c4c28875 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:56:05 UTC) from {username='slave'} at 127.0.0.1:43958
I20260809 08:56:05.414248 6236 tablet_service.cc:1514] Processing DeleteTablet for tablet ca552818b66343f492d11ac1c4c28875 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:56:05 UTC) from {username='slave'} at 127.0.0.1:38486
I20260809 08:56:05.414374 6445 tablet_service.cc:1514] Processing DeleteTablet for tablet ca552818b66343f492d11ac1c4c28875 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:56:05 UTC) from {username='slave'} at 127.0.0.1:44070
I20260809 08:56:05.414413 6618 tablet_replica.cc:330] T ca552818b66343f492d11ac1c4c28875 P 94381e1c6bc14ce781d3efcd6b4e009f: stopping tablet replica
I20260809 08:56:05.414507 6618 raft_consensus.cc:2229] T ca552818b66343f492d11ac1c4c28875 P 94381e1c6bc14ce781d3efcd6b4e009f [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:56:05.414528 6620 tablet_replica.cc:330] T ca552818b66343f492d11ac1c4c28875 P 661d3a8183eb4e10ac2b42580dbedb23: stopping tablet replica
I20260809 08:56:05.414620 6618 raft_consensus.cc:2258] T ca552818b66343f492d11ac1c4c28875 P 94381e1c6bc14ce781d3efcd6b4e009f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:56:05.414731 6620 raft_consensus.cc:2229] T ca552818b66343f492d11ac1c4c28875 P 661d3a8183eb4e10ac2b42580dbedb23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:56:05.414732 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 5926
I20260809 08:56:05.414858 6620 raft_consensus.cc:2258] T ca552818b66343f492d11ac1c4c28875 P 661d3a8183eb4e10ac2b42580dbedb23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:56:05.415413 6622 tablet_replica.cc:330] T ca552818b66343f492d11ac1c4c28875 P f85265cb4a0848ad98fda72d9a20b1f2: stopping tablet replica
I20260809 08:56:05.415539 6622 raft_consensus.cc:2229] T ca552818b66343f492d11ac1c4c28875 P f85265cb4a0848ad98fda72d9a20b1f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:56:05.415633 6622 raft_consensus.cc:2258] T ca552818b66343f492d11ac1c4c28875 P f85265cb4a0848ad98fda72d9a20b1f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:56:05.420667 6622 ts_tablet_manager.cc:1905] T ca552818b66343f492d11ac1c4c28875 P f85265cb4a0848ad98fda72d9a20b1f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:56:05.422066 6620 ts_tablet_manager.cc:1905] T ca552818b66343f492d11ac1c4c28875 P 661d3a8183eb4e10ac2b42580dbedb23: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:56:05.422143 6622 ts_tablet_manager.cc:1918] T ca552818b66343f492d11ac1c4c28875 P f85265cb4a0848ad98fda72d9a20b1f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:56:05.422209 6622 log.cc:1196] T ca552818b66343f492d11ac1c4c28875 P f85265cb4a0848ad98fda72d9a20b1f2: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-2/wal/wals/ca552818b66343f492d11ac1c4c28875
I20260809 08:56:05.422467 6622 ts_tablet_manager.cc:1939] T ca552818b66343f492d11ac1c4c28875 P f85265cb4a0848ad98fda72d9a20b1f2: Deleting consensus metadata
I20260809 08:56:05.423703 6620 ts_tablet_manager.cc:1918] T ca552818b66343f492d11ac1c4c28875 P 661d3a8183eb4e10ac2b42580dbedb23: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:56:05.423768 6620 log.cc:1196] T ca552818b66343f492d11ac1c4c28875 P 661d3a8183eb4e10ac2b42580dbedb23: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265519942966-8808-0/minicluster-data/ts-1/wal/wals/ca552818b66343f492d11ac1c4c28875
I20260809 08:56:05.424016 6620 ts_tablet_manager.cc:1939] T ca552818b66343f492d11ac1c4c28875 P 661d3a8183eb4e10ac2b42580dbedb23: Deleting consensus metadata
I20260809 08:56:05.424487 5815 catalog_manager.cc:4841] TS 661d3a8183eb4e10ac2b42580dbedb23 (127.8.154.2:38729): tablet ca552818b66343f492d11ac1c4c28875 (table test-table [id=5889eae5ecfd4bef89c79ad73bb0e3c7]) successfully deleted
I20260809 08:56:05.425261 5816 catalog_manager.cc:4841] TS f85265cb4a0848ad98fda72d9a20b1f2 (127.8.154.3:45981): tablet ca552818b66343f492d11ac1c4c28875 (table test-table [id=5889eae5ecfd4bef89c79ad73bb0e3c7]) successfully deleted
W20260809 08:56:05.430159 5814 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.154.1:36623 (error 108)
I20260809 08:56:05.430423 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 6130
I20260809 08:56:05.430693 6568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:05.430550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:36623 (local address 127.0.0.1:43964)
0809 08:56:05.430606 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:05.430647 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.1:36623: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 08:56:05.430815 5814 catalog_manager.cc:4584] TS 94381e1c6bc14ce781d3efcd6b4e009f (127.8.154.1:36623): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ca552818b66343f492d11ac1c4c28875: Network error: Client connection negotiation failed: client connection to 127.8.154.1:36623: connect: Connection refused (error 111)
I20260809 08:56:05.443192 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 6337
I20260809 08:56:05.454247 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 5790
2026-08-09T08:56:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (819 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:56:05 dist-test-slave-t3q3 krb5kdc[6627](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:56:05 dist-test-slave-t3q3 krb5kdc[6627](info): set up 2 sockets
Aug 09 08:56:05 dist-test-slave-t3q3 krb5kdc[6627](info): commencing operation
krb5kdc: starting...
W20260809 08:56:06.725926 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:56:06 dist-test-slave-t3q3 krb5kdc[6627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:56:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:56:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
I20260809 08:56:06.853075 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:46545
--webserver_interface=127.8.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:41723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:46545
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:56:06.935667 6644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:06.935860 6644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:06.935882 6644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:06.936966 6644 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:56:06.937002 6644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:06.937017 6644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:06.937029 6644 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:56:06.937041 6644 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:56:06.938424 6644 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:41723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:46545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:46545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:06.938620 6644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:06.938792 6644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:06.943625 6653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:06.943617 6657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:06.943616 6654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:06.943694 6644 server_base.cc:1034] running on GCE node
I20260809 08:56:06.944190 6644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:06.944490 6644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:06.945654 6644 hybrid_clock.cc:648] HybridClock initialized: now 1786265766945646 us; error 44 us; skew 500 ppm
Aug 09 08:56:06 dist-test-slave-t3q3 krb5kdc[6627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265766, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:56:06.948285 6644 init.cc:377] Logged in from keytab as kudu/127.8.154.62@KRBTEST.COM (short username kudu)
I20260809 08:56:06.948761 6644 webserver.cc:460] Webserver started at http://127.8.154.62:41521/ using document root <none> and password file <none>
I20260809 08:56:06.948949 6644 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:06.948990 6644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:06.949087 6644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:06.950091 6644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "de57ab9027b44dd0bbf00daa0e4fa3f5"
format_stamp: "Formatted at 2026-08-09 08:56:06 on dist-test-slave-t3q3"
server_key: "98ccc3bdee7a16698a361358eb809d64"
server_key_iv: "d0597cafb2355a66114faefdb3dc3c8b"
server_key_version: "encryptionkey@0"
I20260809 08:56:06.950405 6644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "de57ab9027b44dd0bbf00daa0e4fa3f5"
format_stamp: "Formatted at 2026-08-09 08:56:06 on dist-test-slave-t3q3"
server_key: "98ccc3bdee7a16698a361358eb809d64"
server_key_iv: "d0597cafb2355a66114faefdb3dc3c8b"
server_key_version: "encryptionkey@0"
I20260809 08:56:06.951774 6644 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:56:06.952589 6667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:06.952749 6644 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:06.952860 6644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "de57ab9027b44dd0bbf00daa0e4fa3f5"
format_stamp: "Formatted at 2026-08-09 08:56:06 on dist-test-slave-t3q3"
server_key: "98ccc3bdee7a16698a361358eb809d64"
server_key_iv: "d0597cafb2355a66114faefdb3dc3c8b"
server_key_version: "encryptionkey@0"
I20260809 08:56:06.952934 6644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:06.962353 6644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:06.964237 6644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:06.964395 6644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:06.968431 6644 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:46545
I20260809 08:56:06.968464 6751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:46545 every 8 connection(s)
I20260809 08:56:06.968819 6644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:56:06.969688 6754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:06.971894 6754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de57ab9027b44dd0bbf00daa0e4fa3f5: Bootstrap starting.
I20260809 08:56:06.972499 6754 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de57ab9027b44dd0bbf00daa0e4fa3f5: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:06.972774 6754 log.cc:824] T 00000000000000000000000000000000 P de57ab9027b44dd0bbf00daa0e4fa3f5: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:06.973489 6754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de57ab9027b44dd0bbf00daa0e4fa3f5: No bootstrap required, opened a new log
I20260809 08:56:06.975065 6754 raft_consensus.cc:348] T 00000000000000000000000000000000 P de57ab9027b44dd0bbf00daa0e4fa3f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de57ab9027b44dd0bbf00daa0e4fa3f5" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46545 } }
I20260809 08:56:06.975208 6754 raft_consensus.cc:374] T 00000000000000000000000000000000 P de57ab9027b44dd0bbf00daa0e4fa3f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:06.975252 6754 raft_consensus.cc:729] T 00000000000000000000000000000000 P de57ab9027b44dd0bbf00daa0e4fa3f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de57ab9027b44dd0bbf00daa0e4fa3f5, State: Initialized, Role: FOLLOWER
I20260809 08:56:06.975390 6754 consensus_queue.cc:260] T 00000000000000000000000000000000 P de57ab9027b44dd0bbf00daa0e4fa3f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de57ab9027b44dd0bbf00daa0e4fa3f5" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46545 } }
I20260809 08:56:06.975451 6754 raft_consensus.cc:388] T 00000000000000000000000000000000 P de57ab9027b44dd0bbf00daa0e4fa3f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:56:06.975488 6754 raft_consensus.cc:482] T 00000000000000000000000000000000 P de57ab9027b44dd0bbf00daa0e4fa3f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:56:06.975538 6754 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de57ab9027b44dd0bbf00daa0e4fa3f5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:06.976125 6754 raft_consensus.cc:504] T 00000000000000000000000000000000 P de57ab9027b44dd0bbf00daa0e4fa3f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de57ab9027b44dd0bbf00daa0e4fa3f5" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46545 } }
I20260809 08:56:06.976251 6754 leader_election.cc:302] T 00000000000000000000000000000000 P de57ab9027b44dd0bbf00daa0e4fa3f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de57ab9027b44dd0bbf00daa0e4fa3f5; no voters:
I20260809 08:56:06.976411 6754 leader_election.cc:288] T 00000000000000000000000000000000 P de57ab9027b44dd0bbf00daa0e4fa3f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:56:06.976469 6757 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de57ab9027b44dd0bbf00daa0e4fa3f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:56:06.976643 6757 raft_consensus.cc:686] T 00000000000000000000000000000000 P de57ab9027b44dd0bbf00daa0e4fa3f5 [term 1 LEADER]: Becoming Leader. State: Replica: de57ab9027b44dd0bbf00daa0e4fa3f5, State: Running, Role: LEADER
I20260809 08:56:06.976678 6754 sys_catalog.cc:564] T 00000000000000000000000000000000 P de57ab9027b44dd0bbf00daa0e4fa3f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:56:06.976745 6757 consensus_queue.cc:237] T 00000000000000000000000000000000 P de57ab9027b44dd0bbf00daa0e4fa3f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de57ab9027b44dd0bbf00daa0e4fa3f5" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46545 } }
I20260809 08:56:06.977164 6760 sys_catalog.cc:455] T 00000000000000000000000000000000 P de57ab9027b44dd0bbf00daa0e4fa3f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader de57ab9027b44dd0bbf00daa0e4fa3f5. Latest consensus state: current_term: 1 leader_uuid: "de57ab9027b44dd0bbf00daa0e4fa3f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de57ab9027b44dd0bbf00daa0e4fa3f5" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46545 } } }
I20260809 08:56:06.977262 6760 sys_catalog.cc:458] T 00000000000000000000000000000000 P de57ab9027b44dd0bbf00daa0e4fa3f5 [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:06.977303 6758 sys_catalog.cc:455] T 00000000000000000000000000000000 P de57ab9027b44dd0bbf00daa0e4fa3f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de57ab9027b44dd0bbf00daa0e4fa3f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de57ab9027b44dd0bbf00daa0e4fa3f5" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 46545 } } }
I20260809 08:56:06.977375 6758 sys_catalog.cc:458] T 00000000000000000000000000000000 P de57ab9027b44dd0bbf00daa0e4fa3f5 [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:06.977502 6768 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:56:06.978067 6768 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:56:06.978385 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 6644
I20260809 08:56:06.978500 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:56:06.978688 8808 external_mini_cluster.cc:1428] Setting key b2e6e997c4503c43a01c3972c1aab74e
I20260809 08:56:06.979745 6768 catalog_manager.cc:1347] Generated new cluster ID: 13183ed7ee1c44999680e6535800eb38
I20260809 08:56:06.979801 6768 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 08:56:06 dist-test-slave-t3q3 krb5kdc[6627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:56:06.990440 6786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:06.979113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50120 (local address 127.8.154.62:46545)
0809 08:56:06.979309 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:06.979317 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:06.979529 (+ 212us) server_negotiation.cc:407] Connection header received
0809 08:56:06.979656 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:06.979663 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:06.979717 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:06.979860 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:06.980650 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:06.982115 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:06.983152 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:06.983776 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:06.983892 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:06.986689 (+ 2797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:06.986694 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:06.986694 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:06.986706 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:06.988664 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:06.989077 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:06.989080 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:06.989081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:06.989197 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:06.989405 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:06.989406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:06.989406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:06.989702 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:06.989852 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:06.989954 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 08:56:06.990195 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":85,"threads_started":1}
I20260809 08:56:06.991254 6768 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:56:06.991736 6768 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:56:06.999946 6768 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de57ab9027b44dd0bbf00daa0e4fa3f5: Generated new TSK 0
I20260809 08:56:07.000146 6768 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.8.154.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab.
Entry for principal kudu/127.8.154.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab.
I20260809 08:56:07.030746 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.1:0
--local_ip_for_outbound_sockets=127.8.154.1
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:46545
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:41723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:56:07.119210 6793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:07.119416 6793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:07.119441 6793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:07.120534 6793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:07.120570 6793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:07.120596 6793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.1
I20260809 08:56:07.121974 6793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:41723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.1.keytab
--principal=kudu/127.8.154.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:46545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.1
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:07.122176 6793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:07.122344 6793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:07.127732 6804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:07.127767 6793 server_base.cc:1034] running on GCE node
W20260809 08:56:07.127768 6800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:07.127733 6799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:07.128290 6793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:07.128505 6793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:07.129788 6793 hybrid_clock.cc:648] HybridClock initialized: now 1786265767129769 us; error 31 us; skew 500 ppm
Aug 09 08:56:07 dist-test-slave-t3q3 krb5kdc[6627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265767, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:56:07.132455 6793 init.cc:377] Logged in from keytab as kudu/127.8.154.1@KRBTEST.COM (short username kudu)
I20260809 08:56:07.133164 6793 webserver.cc:460] Webserver started at http://127.8.154.1:39937/ using document root <none> and password file <none>
I20260809 08:56:07.133438 6793 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:07.133514 6793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:07.133674 6793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:07.134758 6793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-0/data/instance:
uuid: "69d32892acbb4fc4b55e60397bf6e16e"
format_stamp: "Formatted at 2026-08-09 08:56:07 on dist-test-slave-t3q3"
server_key: "36f5fd0ff5b929a74c23fbf2c2f97d99"
server_key_iv: "643684c712f9e7f45a52bf21c1c87f4d"
server_key_version: "encryptionkey@0"
I20260809 08:56:07.135263 6793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance:
uuid: "69d32892acbb4fc4b55e60397bf6e16e"
format_stamp: "Formatted at 2026-08-09 08:56:07 on dist-test-slave-t3q3"
server_key: "36f5fd0ff5b929a74c23fbf2c2f97d99"
server_key_iv: "643684c712f9e7f45a52bf21c1c87f4d"
server_key_version: "encryptionkey@0"
I20260809 08:56:07.136830 6793 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:56:07.137650 6816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:07.137833 6793 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:56:07.137948 6793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-0/wal
uuid: "69d32892acbb4fc4b55e60397bf6e16e"
format_stamp: "Formatted at 2026-08-09 08:56:07 on dist-test-slave-t3q3"
server_key: "36f5fd0ff5b929a74c23fbf2c2f97d99"
server_key_iv: "643684c712f9e7f45a52bf21c1c87f4d"
server_key_version: "encryptionkey@0"
I20260809 08:56:07.138026 6793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:07.152968 6793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:07.154953 6793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:07.155131 6793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:07.155371 6793 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:56:07.155709 6793 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:56:07.155773 6793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:07.155810 6793 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:56:07.155843 6793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:07.161932 6793 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.1:34095
I20260809 08:56:07.161974 6987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.1:34095 every 8 connection(s)
I20260809 08:56:07.162343 6793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
I20260809 08:56:07.166182 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 6793
I20260809 08:56:07.166316 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance
I20260809 08:56:07.166520 8808 external_mini_cluster.cc:1428] Setting key 1cdfd725df93038d6609d1d8e8d357b3
Aug 09 08:56:07 dist-test-slave-t3q3 krb5kdc[6627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265767, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.1@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:56:07.173238 6786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:07.163094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:53379 (local address 127.8.154.62:46545)
0809 08:56:07.163136 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:07.163139 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:07.163380 (+ 241us) server_negotiation.cc:407] Connection header received
0809 08:56:07.163873 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:07.163873 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:07.163886 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:07.163945 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:07.164947 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:07.165470 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:07.166933 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:07.167287 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:07.167369 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:07.169880 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:07.169881 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:07.169881 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:07.169892 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:07.171426 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:07.171971 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:07.171972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:07.171972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:07.172024 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:07.172398 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:07.172399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:07.172399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:07.172511 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:07.172566 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:07.172914 (+ 348us) server_negotiation.cc:299] Negotiation successful
0809 08:56:07.173068 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 08:56:07.173261 6992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:07.163065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46545 (local address 127.8.154.1:53379)
0809 08:56:07.163242 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:07.163284 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:07.163776 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:07.164013 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:07.164016 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:07.164629 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:56:07.164830 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:07.164835 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:07.165584 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:07.165585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:07.166780 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:07.166784 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:07.167386 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:07.167387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:07.167581 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:07.168112 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:56:07.168146 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:56:07.169776 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:56:07.171516 (+ 1740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:07.171525 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:07.171528 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:07.171886 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:07.172172 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:07.172173 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:07.172173 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:07.172323 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:07.172587 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:56:07.172594 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:56:07.172768 (+ 174us) client_negotiation.cc:769] Sending connection context
0809 08:56:07.172865 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 08:56:07.173005 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":101,"threads_started":1}
I20260809 08:56:07.173854 6988 heartbeater.cc:347] Connected to a master server at 127.8.154.62:46545
I20260809 08:56:07.173976 6988 heartbeater.cc:464] Registering TS with master...
I20260809 08:56:07.174206 6988 heartbeater.cc:511] Master 127.8.154.62:46545 requested a full tablet report, sending...
I20260809 08:56:07.174806 6689 ts_manager.cc:194] Registered new tserver with Master: 69d32892acbb4fc4b55e60397bf6e16e (127.8.154.1:34095)
I20260809 08:56:07.175441 6689 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.1@KRBTEST.COM'} at 127.8.154.1:53379
WARNING: no policy specified for kudu/127.8.154.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab.
Entry for principal kudu/127.8.154.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab.
I20260809 08:56:07.195622 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.2:0
--local_ip_for_outbound_sockets=127.8.154.2
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:46545
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:41723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:56:07.281662 6997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:07.281868 6997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:07.281891 6997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:07.283041 6997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:07.283082 6997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:07.283107 6997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.2
I20260809 08:56:07.284577 6997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:41723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.2.keytab
--principal=kudu/127.8.154.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:46545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.2
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:07.284821 6997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:07.285076 6997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:07.290256 7011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:07.290267 7012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:07.290351 7015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:07.290664 6997 server_base.cc:1034] running on GCE node
I20260809 08:56:07.290900 6997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:07.291193 6997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:07.292336 6997 hybrid_clock.cc:648] HybridClock initialized: now 1786265767292320 us; error 32 us; skew 500 ppm
Aug 09 08:56:07 dist-test-slave-t3q3 krb5kdc[6627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265767, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:56:07.294938 6997 init.cc:377] Logged in from keytab as kudu/127.8.154.2@KRBTEST.COM (short username kudu)
I20260809 08:56:07.295434 6997 webserver.cc:460] Webserver started at http://127.8.154.2:44737/ using document root <none> and password file <none>
I20260809 08:56:07.295686 6997 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:07.295758 6997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:07.295909 6997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:07.297008 6997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-1/data/instance:
uuid: "4b7c2656c39d4700b584200bfcc1a115"
format_stamp: "Formatted at 2026-08-09 08:56:07 on dist-test-slave-t3q3"
server_key: "1bfff676e285b0082eda03a7c11e4a59"
server_key_iv: "ed0f786377ba87503b16ff7755e997be"
server_key_version: "encryptionkey@0"
I20260809 08:56:07.297391 6997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance:
uuid: "4b7c2656c39d4700b584200bfcc1a115"
format_stamp: "Formatted at 2026-08-09 08:56:07 on dist-test-slave-t3q3"
server_key: "1bfff676e285b0082eda03a7c11e4a59"
server_key_iv: "ed0f786377ba87503b16ff7755e997be"
server_key_version: "encryptionkey@0"
I20260809 08:56:07.298743 6997 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:56:07.299506 7028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:07.299702 6997 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:56:07.299794 6997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-1/wal
uuid: "4b7c2656c39d4700b584200bfcc1a115"
format_stamp: "Formatted at 2026-08-09 08:56:07 on dist-test-slave-t3q3"
server_key: "1bfff676e285b0082eda03a7c11e4a59"
server_key_iv: "ed0f786377ba87503b16ff7755e997be"
server_key_version: "encryptionkey@0"
I20260809 08:56:07.299876 6997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:07.315029 6997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:07.317241 6997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:07.317430 6997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:07.317694 6997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:56:07.318037 6997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:56:07.318092 6997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:07.318135 6997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:56:07.318177 6997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:07.324952 6997 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.2:44053
I20260809 08:56:07.325001 7196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.2:44053 every 8 connection(s)
I20260809 08:56:07.325345 6997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
Aug 09 08:56:07 dist-test-slave-t3q3 krb5kdc[6627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265767, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.2@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:56:07.331795 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 6997
I20260809 08:56:07.331907 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance
I20260809 08:56:07.332155 8808 external_mini_cluster.cc:1428] Setting key 31d5dc5cc8af9a2204f0298deb346073
I20260809 08:56:07.335552 7200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:07.326060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46545 (local address 127.8.154.2:35181)
0809 08:56:07.326237 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:07.326274 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:07.326796 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:07.327097 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:07.327101 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:07.327681 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:56:07.327881 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:07.327885 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:07.328673 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:07.328674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:07.329515 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:07.329517 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:07.329947 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:07.329948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:07.330080 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:07.330542 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:56:07.330550 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:56:07.332022 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:56:07.333936 (+ 1914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:07.333944 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:07.333946 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:07.334265 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:07.334543 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:07.334544 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:07.334544 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:07.334700 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:07.335052 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:56:07.335058 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:56:07.335182 (+ 124us) client_negotiation.cc:769] Sending connection context
0809 08:56:07.335272 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 08:56:07.335392 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":100,"threads_started":1}
I20260809 08:56:07.335551 6786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:07.326086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:35181 (local address 127.8.154.62:46545)
0809 08:56:07.326153 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:07.326155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:07.326368 (+ 213us) server_negotiation.cc:407] Connection header received
0809 08:56:07.326910 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:07.326911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:07.326925 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:07.326991 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:07.327993 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:07.328567 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:07.329615 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:07.329842 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:07.329935 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:07.332177 (+ 2242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:07.332179 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:07.332179 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:07.332187 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:07.333852 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:07.334406 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:07.334407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:07.334407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:07.334467 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:07.334800 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:07.334802 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:07.334802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:07.334975 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:07.335056 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:07.335319 (+ 263us) server_negotiation.cc:299] Negotiation successful
0809 08:56:07.335415 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 08:56:07.336133 7198 heartbeater.cc:347] Connected to a master server at 127.8.154.62:46545
I20260809 08:56:07.336249 7198 heartbeater.cc:464] Registering TS with master...
I20260809 08:56:07.336453 7198 heartbeater.cc:511] Master 127.8.154.62:46545 requested a full tablet report, sending...
I20260809 08:56:07.336846 6689 ts_manager.cc:194] Registered new tserver with Master: 4b7c2656c39d4700b584200bfcc1a115 (127.8.154.2:44053)
I20260809 08:56:07.337237 6689 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.2@KRBTEST.COM'} at 127.8.154.2:35181
WARNING: no policy specified for kudu/127.8.154.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab.
Entry for principal kudu/127.8.154.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab.
I20260809 08:56:07.358718 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.3:0
--local_ip_for_outbound_sockets=127.8.154.3
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:46545
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:41723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:56:07.442180 7208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:07.442400 7208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:07.442426 7208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:07.443598 7208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:07.443643 7208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:07.443672 7208 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.3
I20260809 08:56:07.445282 7208 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:41723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.3.keytab
--principal=kudu/127.8.154.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:46545
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.3
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:07.445555 7208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:07.445801 7208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:07.451143 7222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:07.451210 7218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:07.451336 7208 server_base.cc:1034] running on GCE node
W20260809 08:56:07.451164 7219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:07.451586 7208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:07.451892 7208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:07.453065 7208 hybrid_clock.cc:648] HybridClock initialized: now 1786265767453049 us; error 33 us; skew 500 ppm
Aug 09 08:56:07 dist-test-slave-t3q3 krb5kdc[6627](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265767, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:56:07.455667 7208 init.cc:377] Logged in from keytab as kudu/127.8.154.3@KRBTEST.COM (short username kudu)
I20260809 08:56:07.456210 7208 webserver.cc:460] Webserver started at http://127.8.154.3:40793/ using document root <none> and password file <none>
I20260809 08:56:07.456461 7208 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:07.456507 7208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:07.456604 7208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:07.457639 7208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-2/data/instance:
uuid: "d1a1c90b5feb493fa0baca4b889ee07e"
format_stamp: "Formatted at 2026-08-09 08:56:07 on dist-test-slave-t3q3"
server_key: "f6f11449891251713e5781ccc4fb393d"
server_key_iv: "4010e3589b7c9824bd69de39c2a1105a"
server_key_version: "encryptionkey@0"
I20260809 08:56:07.457957 7208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance:
uuid: "d1a1c90b5feb493fa0baca4b889ee07e"
format_stamp: "Formatted at 2026-08-09 08:56:07 on dist-test-slave-t3q3"
server_key: "f6f11449891251713e5781ccc4fb393d"
server_key_iv: "4010e3589b7c9824bd69de39c2a1105a"
server_key_version: "encryptionkey@0"
I20260809 08:56:07.459246 7208 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:56:07.459919 7232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:07.460135 7208 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:56:07.460233 7208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-2/wal
uuid: "d1a1c90b5feb493fa0baca4b889ee07e"
format_stamp: "Formatted at 2026-08-09 08:56:07 on dist-test-slave-t3q3"
server_key: "f6f11449891251713e5781ccc4fb393d"
server_key_iv: "4010e3589b7c9824bd69de39c2a1105a"
server_key_version: "encryptionkey@0"
I20260809 08:56:07.460306 7208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:07.489818 7208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:07.492159 7208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:07.492281 7208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:07.492471 7208 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:56:07.492812 7208 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:56:07.492847 7208 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:07.492902 7208 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:56:07.492930 7208 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:07.498831 7208 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.3:37399
I20260809 08:56:07.498939 7418 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.3:37399 every 8 connection(s)
I20260809 08:56:07.499257 7208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
I20260809 08:56:07.504186 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 7208
I20260809 08:56:07.504315 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance
I20260809 08:56:07.504523 8808 external_mini_cluster.cc:1428] Setting key dcdb3e63a3387b5b147dabe6eed11317
Aug 09 08:56:07 dist-test-slave-t3q3 krb5kdc[6627](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265767, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.3@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:56:07.509279 6786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:07.500049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:34807 (local address 127.8.154.62:46545)
0809 08:56:07.500138 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:07.500140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:07.500316 (+ 176us) server_negotiation.cc:407] Connection header received
0809 08:56:07.500873 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:07.500875 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:07.500888 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:07.500939 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:07.501916 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:07.502438 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:07.503490 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:07.503715 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:07.503789 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:07.506381 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:07.506382 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:07.506383 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:07.506390 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:07.507743 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:07.508263 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:07.508265 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:07.508266 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:07.508329 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:07.508615 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:07.508616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:07.508616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:07.508734 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:07.508807 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:07.509106 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 08:56:07.509190 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 08:56:07.509469 7424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:07.499966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:46545 (local address 127.8.154.3:34807)
0809 08:56:07.500174 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:07.500226 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:07.500757 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:07.501010 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:07.501014 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:07.501609 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:56:07.501810 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:07.501816 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:07.502538 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:07.502539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:07.503395 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:07.503398 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:07.503810 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:07.503811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:07.503942 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:07.504525 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:56:07.504533 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:56:07.506266 (+ 1733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:56:07.507858 (+ 1592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:07.507866 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:07.507868 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:07.508170 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:07.508406 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:07.508407 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:07.508407 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:07.508531 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:07.508811 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:56:07.508818 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:56:07.508959 (+ 141us) client_negotiation.cc:769] Sending connection context
0809 08:56:07.509060 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 08:56:07.509219 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":103,"threads_started":1}
I20260809 08:56:07.510006 7419 heartbeater.cc:347] Connected to a master server at 127.8.154.62:46545
I20260809 08:56:07.510131 7419 heartbeater.cc:464] Registering TS with master...
I20260809 08:56:07.510314 7419 heartbeater.cc:511] Master 127.8.154.62:46545 requested a full tablet report, sending...
I20260809 08:56:07.510705 6689 ts_manager.cc:194] Registered new tserver with Master: d1a1c90b5feb493fa0baca4b889ee07e (127.8.154.3:37399)
I20260809 08:56:07.511040 6689 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.8.154.3@KRBTEST.COM'} at 127.8.154.3:34807
I20260809 08:56:07.516599 8808 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:56:07.523664 6786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:07.517939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50126 (local address 127.8.154.62:46545)
0809 08:56:07.517999 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:07.518001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:07.518107 (+ 106us) server_negotiation.cc:407] Connection header received
0809 08:56:07.518159 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:07.518160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:07.518168 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:07.518222 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:07.518740 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:07.519219 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:07.519944 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:07.520205 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:07.520261 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:07.521366 (+ 1105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:07.521367 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:07.521368 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:07.521374 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:07.522624 (+ 1250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:07.522953 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:07.522953 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:07.522954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:07.523005 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:07.523229 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:07.523229 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:07.523229 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:07.523363 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:07.523441 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:07.523543 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 08:56:07.523605 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 08:56:07.525238 6689 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50126:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:56:07.526757 6689 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:56:07.533723 7441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:07.529620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56926 (local address 127.8.154.3:37399)
0809 08:56:07.529850 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:07.529854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:07.529864 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:56:07.529908 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:07.529910 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:07.529925 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:07.530010 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:56:07.530175 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:07.530969 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:07.532552 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:07.533253 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:07.533335 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:07.533395 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:07.533591 (+ 196us) server_negotiation.cc:299] Negotiation successful
0809 08:56:07.533635 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":93,"threads_started":1}
I20260809 08:56:07.533718 7437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:07.529485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:37399 (local address 127.0.0.1:56926)
0809 08:56:07.529771 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:07.529782 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:07.529825 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:07.530004 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:07.530005 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:07.530007 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:56:07.530111 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:07.530114 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:07.531085 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:07.531086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:07.532463 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:07.532467 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:07.533360 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:07.533361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:07.533457 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:07.533470 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 08:56:07.533558 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 08:56:07.533622 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"mutex_wait_us":70,"thread_start_us":77,"threads_started":1}
I20260809 08:56:07.533717 7440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:07.529285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:34095 (local address 127.0.0.1:43082)
0809 08:56:07.529604 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:07.529614 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:07.529665 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:07.530312 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:07.530313 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:07.530315 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:56:07.530351 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:07.530353 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:07.531484 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:07.531485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:07.532394 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:07.532397 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:07.533358 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:07.533360 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:07.533455 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:07.533470 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 08:56:07.533558 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 08:56:07.533622 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":90,"threads_started":1}
I20260809 08:56:07.534363 7439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:07.529428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43082 (local address 127.8.154.1:34095)
0809 08:56:07.530145 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:07.530149 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:07.530159 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:56:07.530197 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:07.530199 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:07.530213 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:07.530294 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:56:07.530419 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:07.531378 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:07.532502 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:07.533253 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:07.533335 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:07.533394 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:07.534226 (+ 832us) server_negotiation.cc:299] Negotiation successful
0809 08:56:07.534278 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":679,"thread_start_us":91,"threads_started":1}
I20260809 08:56:07.534608 7436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:07.529089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:44053 (local address 127.0.0.1:45456)
0809 08:56:07.529203 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:07.529246 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:07.529330 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:07.529666 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:07.529670 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:07.529672 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:56:07.529756 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:07.529759 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:07.531131 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:07.531131 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:07.532485 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:07.532487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:07.534334 (+ 1847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:07.534336 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:07.534403 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:07.534407 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:56:07.534467 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 08:56:07.534537 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"mutex_wait_us":53,"thread_start_us":44,"threads_started":1}
I20260809 08:56:07.535130 7310 tablet_service.cc:1467] Processing CreateTablet for tablet 0a295f76cdb24591992511d46af4e212 (DEFAULT_TABLE table=test-table [id=fba5146be7c4446ead9b046a6c01c147]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:56:07.535130 6883 tablet_service.cc:1467] Processing CreateTablet for tablet 0a295f76cdb24591992511d46af4e212 (DEFAULT_TABLE table=test-table [id=fba5146be7c4446ead9b046a6c01c147]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:56:07.535496 6883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a295f76cdb24591992511d46af4e212. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:07.535504 7310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a295f76cdb24591992511d46af4e212. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:07.536303 7438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:07.529272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45456 (local address 127.8.154.2:44053)
0809 08:56:07.529494 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:07.529499 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:07.529511 (+ 12us) server_negotiation.cc:407] Connection header received
0809 08:56:07.529557 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:07.529560 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:07.529576 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:07.529660 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:56:07.529854 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:07.530985 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:07.533434 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:07.534242 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:07.534317 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:07.534374 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:07.536201 (+ 1827us) server_negotiation.cc:299] Negotiation successful
0809 08:56:07.536241 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":96,"threads_started":1}
I20260809 08:56:07.536872 7095 tablet_service.cc:1467] Processing CreateTablet for tablet 0a295f76cdb24591992511d46af4e212 (DEFAULT_TABLE table=test-table [id=fba5146be7c4446ead9b046a6c01c147]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:56:07.537132 7095 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a295f76cdb24591992511d46af4e212. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:07.538975 7442 tablet_bootstrap.cc:492] T 0a295f76cdb24591992511d46af4e212 P 69d32892acbb4fc4b55e60397bf6e16e: Bootstrap starting.
I20260809 08:56:07.539232 7443 tablet_bootstrap.cc:492] T 0a295f76cdb24591992511d46af4e212 P d1a1c90b5feb493fa0baca4b889ee07e: Bootstrap starting.
I20260809 08:56:07.539933 7443 tablet_bootstrap.cc:654] T 0a295f76cdb24591992511d46af4e212 P d1a1c90b5feb493fa0baca4b889ee07e: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:07.540340 7444 tablet_bootstrap.cc:492] T 0a295f76cdb24591992511d46af4e212 P 4b7c2656c39d4700b584200bfcc1a115: Bootstrap starting.
I20260809 08:56:07.540829 7443 log.cc:824] T 0a295f76cdb24591992511d46af4e212 P d1a1c90b5feb493fa0baca4b889ee07e: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:07.540963 7442 tablet_bootstrap.cc:654] T 0a295f76cdb24591992511d46af4e212 P 69d32892acbb4fc4b55e60397bf6e16e: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:07.540963 7444 tablet_bootstrap.cc:654] T 0a295f76cdb24591992511d46af4e212 P 4b7c2656c39d4700b584200bfcc1a115: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:07.541613 7443 tablet_bootstrap.cc:492] T 0a295f76cdb24591992511d46af4e212 P d1a1c90b5feb493fa0baca4b889ee07e: No bootstrap required, opened a new log
I20260809 08:56:07.541654 7444 log.cc:824] T 0a295f76cdb24591992511d46af4e212 P 4b7c2656c39d4700b584200bfcc1a115: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:07.541658 7442 log.cc:824] T 0a295f76cdb24591992511d46af4e212 P 69d32892acbb4fc4b55e60397bf6e16e: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:07.541698 7443 ts_tablet_manager.cc:1397] T 0a295f76cdb24591992511d46af4e212 P d1a1c90b5feb493fa0baca4b889ee07e: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260809 08:56:07.542435 7444 tablet_bootstrap.cc:492] T 0a295f76cdb24591992511d46af4e212 P 4b7c2656c39d4700b584200bfcc1a115: No bootstrap required, opened a new log
I20260809 08:56:07.542510 7444 ts_tablet_manager.cc:1397] T 0a295f76cdb24591992511d46af4e212 P 4b7c2656c39d4700b584200bfcc1a115: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:56:07.542713 7442 tablet_bootstrap.cc:492] T 0a295f76cdb24591992511d46af4e212 P 69d32892acbb4fc4b55e60397bf6e16e: No bootstrap required, opened a new log
I20260809 08:56:07.542788 7442 ts_tablet_manager.cc:1397] T 0a295f76cdb24591992511d46af4e212 P 69d32892acbb4fc4b55e60397bf6e16e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 08:56:07.543479 7443 raft_consensus.cc:348] T 0a295f76cdb24591992511d46af4e212 P d1a1c90b5feb493fa0baca4b889ee07e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7c2656c39d4700b584200bfcc1a115" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 44053 } } peers { permanent_uuid: "69d32892acbb4fc4b55e60397bf6e16e" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 34095 } } peers { permanent_uuid: "d1a1c90b5feb493fa0baca4b889ee07e" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 37399 } }
I20260809 08:56:07.543648 7443 raft_consensus.cc:374] T 0a295f76cdb24591992511d46af4e212 P d1a1c90b5feb493fa0baca4b889ee07e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:07.543697 7443 raft_consensus.cc:729] T 0a295f76cdb24591992511d46af4e212 P d1a1c90b5feb493fa0baca4b889ee07e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1a1c90b5feb493fa0baca4b889ee07e, State: Initialized, Role: FOLLOWER
I20260809 08:56:07.543810 7443 consensus_queue.cc:260] T 0a295f76cdb24591992511d46af4e212 P d1a1c90b5feb493fa0baca4b889ee07e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7c2656c39d4700b584200bfcc1a115" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 44053 } } peers { permanent_uuid: "69d32892acbb4fc4b55e60397bf6e16e" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 34095 } } peers { permanent_uuid: "d1a1c90b5feb493fa0baca4b889ee07e" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 37399 } }
I20260809 08:56:07.544142 7443 ts_tablet_manager.cc:1428] T 0a295f76cdb24591992511d46af4e212 P d1a1c90b5feb493fa0baca4b889ee07e: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 08:56:07.544160 7419 heartbeater.cc:503] Master 127.8.154.62:46545 was elected leader, sending a full tablet report...
I20260809 08:56:07.544828 7442 raft_consensus.cc:348] T 0a295f76cdb24591992511d46af4e212 P 69d32892acbb4fc4b55e60397bf6e16e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7c2656c39d4700b584200bfcc1a115" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 44053 } } peers { permanent_uuid: "69d32892acbb4fc4b55e60397bf6e16e" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 34095 } } peers { permanent_uuid: "d1a1c90b5feb493fa0baca4b889ee07e" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 37399 } }
I20260809 08:56:07.545015 7442 raft_consensus.cc:374] T 0a295f76cdb24591992511d46af4e212 P 69d32892acbb4fc4b55e60397bf6e16e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:07.545061 7442 raft_consensus.cc:729] T 0a295f76cdb24591992511d46af4e212 P 69d32892acbb4fc4b55e60397bf6e16e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69d32892acbb4fc4b55e60397bf6e16e, State: Initialized, Role: FOLLOWER
I20260809 08:56:07.545183 7442 consensus_queue.cc:260] T 0a295f76cdb24591992511d46af4e212 P 69d32892acbb4fc4b55e60397bf6e16e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7c2656c39d4700b584200bfcc1a115" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 44053 } } peers { permanent_uuid: "69d32892acbb4fc4b55e60397bf6e16e" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 34095 } } peers { permanent_uuid: "d1a1c90b5feb493fa0baca4b889ee07e" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 37399 } }
I20260809 08:56:07.544883 7444 raft_consensus.cc:348] T 0a295f76cdb24591992511d46af4e212 P 4b7c2656c39d4700b584200bfcc1a115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7c2656c39d4700b584200bfcc1a115" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 44053 } } peers { permanent_uuid: "69d32892acbb4fc4b55e60397bf6e16e" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 34095 } } peers { permanent_uuid: "d1a1c90b5feb493fa0baca4b889ee07e" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 37399 } }
I20260809 08:56:07.545424 7444 raft_consensus.cc:374] T 0a295f76cdb24591992511d46af4e212 P 4b7c2656c39d4700b584200bfcc1a115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:07.545470 7444 raft_consensus.cc:729] T 0a295f76cdb24591992511d46af4e212 P 4b7c2656c39d4700b584200bfcc1a115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b7c2656c39d4700b584200bfcc1a115, State: Initialized, Role: FOLLOWER
I20260809 08:56:07.545512 7442 ts_tablet_manager.cc:1428] T 0a295f76cdb24591992511d46af4e212 P 69d32892acbb4fc4b55e60397bf6e16e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:56:07.545547 6988 heartbeater.cc:503] Master 127.8.154.62:46545 was elected leader, sending a full tablet report...
I20260809 08:56:07.545646 7444 consensus_queue.cc:260] T 0a295f76cdb24591992511d46af4e212 P 4b7c2656c39d4700b584200bfcc1a115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7c2656c39d4700b584200bfcc1a115" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 44053 } } peers { permanent_uuid: "69d32892acbb4fc4b55e60397bf6e16e" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 34095 } } peers { permanent_uuid: "d1a1c90b5feb493fa0baca4b889ee07e" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 37399 } }
I20260809 08:56:07.545948 7444 ts_tablet_manager.cc:1428] T 0a295f76cdb24591992511d46af4e212 P 4b7c2656c39d4700b584200bfcc1a115: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:56:07.546119 7198 heartbeater.cc:503] Master 127.8.154.62:46545 was elected leader, sending a full tablet report...
W20260809 08:56:07.575928 7199 tablet.cc:2369] T 0a295f76cdb24591992511d46af4e212 P 4b7c2656c39d4700b584200bfcc1a115: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 08:56:07.663151 6990 tablet.cc:2369] T 0a295f76cdb24591992511d46af4e212 P 69d32892acbb4fc4b55e60397bf6e16e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:56:07.694774 7454 raft_consensus.cc:482] T 0a295f76cdb24591992511d46af4e212 P 69d32892acbb4fc4b55e60397bf6e16e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:56:07.694937 7454 raft_consensus.cc:504] T 0a295f76cdb24591992511d46af4e212 P 69d32892acbb4fc4b55e60397bf6e16e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7c2656c39d4700b584200bfcc1a115" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 44053 } } peers { permanent_uuid: "69d32892acbb4fc4b55e60397bf6e16e" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 34095 } } peers { permanent_uuid: "d1a1c90b5feb493fa0baca4b889ee07e" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 37399 } }
I20260809 08:56:07.695333 7454 leader_election.cc:288] T 0a295f76cdb24591992511d46af4e212 P 69d32892acbb4fc4b55e60397bf6e16e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b7c2656c39d4700b584200bfcc1a115 (127.8.154.2:44053), d1a1c90b5feb493fa0baca4b889ee07e (127.8.154.3:37399)
I20260809 08:56:07.698522 7438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:07.695686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:54385 (local address 127.8.154.2:44053)
0809 08:56:07.695750 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:07.695752 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:07.695803 (+ 51us) server_negotiation.cc:407] Connection header received
0809 08:56:07.695869 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:07.695870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:07.695884 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:07.695968 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:56:07.696226 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:07.696757 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:07.697753 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:07.698160 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:07.698217 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:07.698226 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:07.698399 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 08:56:07.698448 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 08:56:07.698549 7456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:07.695585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:44053 (local address 127.8.154.1:54385)
0809 08:56:07.695720 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:07.695734 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:07.695804 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:07.696000 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:07.696002 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:07.696005 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:56:07.696145 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:07.696147 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:07.696844 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:07.696845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:07.697660 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:07.697663 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:07.698252 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:07.698253 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:07.698329 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:07.698333 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:56:07.698376 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 08:56:07.698427 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":86,"threads_started":1}
I20260809 08:56:07.698570 7441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:07.695948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:58823 (local address 127.8.154.3:37399)
0809 08:56:07.696003 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:07.696005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:07.696014 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:56:07.696040 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:07.696041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:07.696051 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:07.696133 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:56:07.696341 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:07.696841 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:07.697797 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:07.698174 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:07.698268 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:07.698278 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:07.698453 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 08:56:07.698495 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 08:56:07.698764 7459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:07.695797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:37399 (local address 127.8.154.1:58823)
0809 08:56:07.695905 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:07.695917 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:07.695968 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:07.696149 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:07.696150 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:07.696174 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:56:07.696265 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:07.696267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:07.696926 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:07.696927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:07.697675 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:07.697677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:07.698284 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:07.698285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:07.698353 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:07.698357 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:56:07.698433 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 08:56:07.698468 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":43,"threads_started":1}
I20260809 08:56:07.699014 7129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a295f76cdb24591992511d46af4e212" candidate_uuid: "69d32892acbb4fc4b55e60397bf6e16e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b7c2656c39d4700b584200bfcc1a115" is_pre_election: true
I20260809 08:56:07.699173 7129 raft_consensus.cc:2454] T 0a295f76cdb24591992511d46af4e212 P 4b7c2656c39d4700b584200bfcc1a115 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69d32892acbb4fc4b55e60397bf6e16e in term 0.
I20260809 08:56:07.699235 7347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a295f76cdb24591992511d46af4e212" candidate_uuid: "69d32892acbb4fc4b55e60397bf6e16e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1a1c90b5feb493fa0baca4b889ee07e" is_pre_election: true
I20260809 08:56:07.699406 7347 raft_consensus.cc:2454] T 0a295f76cdb24591992511d46af4e212 P d1a1c90b5feb493fa0baca4b889ee07e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 69d32892acbb4fc4b55e60397bf6e16e in term 0.
I20260809 08:56:07.699429 6818 leader_election.cc:302] T 0a295f76cdb24591992511d46af4e212 P 69d32892acbb4fc4b55e60397bf6e16e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b7c2656c39d4700b584200bfcc1a115, 69d32892acbb4fc4b55e60397bf6e16e; no voters:
I20260809 08:56:07.699569 7454 raft_consensus.cc:2781] T 0a295f76cdb24591992511d46af4e212 P 69d32892acbb4fc4b55e60397bf6e16e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:56:07.699646 7454 raft_consensus.cc:482] T 0a295f76cdb24591992511d46af4e212 P 69d32892acbb4fc4b55e60397bf6e16e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:56:07.699678 7454 raft_consensus.cc:3037] T 0a295f76cdb24591992511d46af4e212 P 69d32892acbb4fc4b55e60397bf6e16e [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:07.700526 7454 raft_consensus.cc:504] T 0a295f76cdb24591992511d46af4e212 P 69d32892acbb4fc4b55e60397bf6e16e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7c2656c39d4700b584200bfcc1a115" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 44053 } } peers { permanent_uuid: "69d32892acbb4fc4b55e60397bf6e16e" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 34095 } } peers { permanent_uuid: "d1a1c90b5feb493fa0baca4b889ee07e" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 37399 } }
I20260809 08:56:07.700668 7454 leader_election.cc:288] T 0a295f76cdb24591992511d46af4e212 P 69d32892acbb4fc4b55e60397bf6e16e [CANDIDATE]: Term 1 election: Requested vote from peers 4b7c2656c39d4700b584200bfcc1a115 (127.8.154.2:44053), d1a1c90b5feb493fa0baca4b889ee07e (127.8.154.3:37399)
I20260809 08:56:07.700872 7129 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a295f76cdb24591992511d46af4e212" candidate_uuid: "69d32892acbb4fc4b55e60397bf6e16e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b7c2656c39d4700b584200bfcc1a115"
I20260809 08:56:07.700904 7347 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a295f76cdb24591992511d46af4e212" candidate_uuid: "69d32892acbb4fc4b55e60397bf6e16e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1a1c90b5feb493fa0baca4b889ee07e"
I20260809 08:56:07.700973 7347 raft_consensus.cc:3037] T 0a295f76cdb24591992511d46af4e212 P d1a1c90b5feb493fa0baca4b889ee07e [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:07.700978 7129 raft_consensus.cc:3037] T 0a295f76cdb24591992511d46af4e212 P 4b7c2656c39d4700b584200bfcc1a115 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:07.701727 7347 raft_consensus.cc:2454] T 0a295f76cdb24591992511d46af4e212 P d1a1c90b5feb493fa0baca4b889ee07e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69d32892acbb4fc4b55e60397bf6e16e in term 1.
I20260809 08:56:07.701753 7129 raft_consensus.cc:2454] T 0a295f76cdb24591992511d46af4e212 P 4b7c2656c39d4700b584200bfcc1a115 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 69d32892acbb4fc4b55e60397bf6e16e in term 1.
I20260809 08:56:07.701949 6818 leader_election.cc:302] T 0a295f76cdb24591992511d46af4e212 P 69d32892acbb4fc4b55e60397bf6e16e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b7c2656c39d4700b584200bfcc1a115, 69d32892acbb4fc4b55e60397bf6e16e; no voters:
I20260809 08:56:07.702066 7454 raft_consensus.cc:2781] T 0a295f76cdb24591992511d46af4e212 P 69d32892acbb4fc4b55e60397bf6e16e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:56:07.702315 7454 raft_consensus.cc:686] T 0a295f76cdb24591992511d46af4e212 P 69d32892acbb4fc4b55e60397bf6e16e [term 1 LEADER]: Becoming Leader. State: Replica: 69d32892acbb4fc4b55e60397bf6e16e, State: Running, Role: LEADER
I20260809 08:56:07.702440 7454 consensus_queue.cc:237] T 0a295f76cdb24591992511d46af4e212 P 69d32892acbb4fc4b55e60397bf6e16e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7c2656c39d4700b584200bfcc1a115" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 44053 } } peers { permanent_uuid: "69d32892acbb4fc4b55e60397bf6e16e" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 34095 } } peers { permanent_uuid: "d1a1c90b5feb493fa0baca4b889ee07e" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 37399 } }
I20260809 08:56:07.703466 6687 catalog_manager.cc:5495] T 0a295f76cdb24591992511d46af4e212 P 69d32892acbb4fc4b55e60397bf6e16e reported cstate change: term changed from 0 to 1, leader changed from <none> to 69d32892acbb4fc4b55e60397bf6e16e (127.8.154.1). New cstate: current_term: 1 leader_uuid: "69d32892acbb4fc4b55e60397bf6e16e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7c2656c39d4700b584200bfcc1a115" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 44053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "69d32892acbb4fc4b55e60397bf6e16e" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 34095 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1a1c90b5feb493fa0baca4b889ee07e" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 37399 } health_report { overall_health: UNKNOWN } } }
I20260809 08:56:07.714378 7439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:07.712137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43092 (local address 127.8.154.1:34095)
0809 08:56:07.712185 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:07.712187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:07.712195 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:56:07.712274 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:07.712275 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:07.712297 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:07.712362 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:56:07.712491 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:07.712954 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:07.713688 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:07.713869 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:07.713921 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:07.714070 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:56:07.714184 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:56:07.714247 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:07.714280 (+ 33us) server_negotiation.cc:299] Negotiation successful
0809 08:56:07.714296 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 08:56:07.716357 7129 raft_consensus.cc:1261] T 0a295f76cdb24591992511d46af4e212 P 4b7c2656c39d4700b584200bfcc1a115 [term 1 FOLLOWER]: Refusing update from remote peer 69d32892acbb4fc4b55e60397bf6e16e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:56:07.716404 7347 raft_consensus.cc:1261] T 0a295f76cdb24591992511d46af4e212 P d1a1c90b5feb493fa0baca4b889ee07e [term 1 FOLLOWER]: Refusing update from remote peer 69d32892acbb4fc4b55e60397bf6e16e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:56:07.716807 7461 consensus_queue.cc:1035] T 0a295f76cdb24591992511d46af4e212 P 69d32892acbb4fc4b55e60397bf6e16e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1a1c90b5feb493fa0baca4b889ee07e" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 37399 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:56:07.716933 7454 consensus_queue.cc:1035] T 0a295f76cdb24591992511d46af4e212 P 69d32892acbb4fc4b55e60397bf6e16e [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b7c2656c39d4700b584200bfcc1a115" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 44053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:56:07.719733 7464 mvcc.cc:205] Tried to move back new op lower bound from 7316544584563183616 to 7316544584509874176. Current Snapshot: MvccSnapshot[applied={T|T < 7316544584563183616}]
I20260809 08:56:07.720098 7465 mvcc.cc:205] Tried to move back new op lower bound from 7316544584563183616 to 7316544584509874176. Current Snapshot: MvccSnapshot[applied={T|T < 7316544584563183616}]
I20260809 08:56:07.720968 7467 mvcc.cc:205] Tried to move back new op lower bound from 7316544584563183616 to 7316544584509874176. Current Snapshot: MvccSnapshot[applied={T|T < 7316544584563183616 or (T in {7316544584563183616})}]
I20260809 08:56:07.722249 6689 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50126:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:56:07.722345 6689 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50126:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:56:07.723155 6689 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de57ab9027b44dd0bbf00daa0e4fa3f5: Sending DeleteTablet for 3 replicas of tablet 0a295f76cdb24591992511d46af4e212
I20260809 08:56:07.723516 7095 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a295f76cdb24591992511d46af4e212 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:56:07 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:45456
I20260809 08:56:07.723552 6883 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a295f76cdb24591992511d46af4e212 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:56:07 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:43082
I20260809 08:56:07.723753 7479 tablet_replica.cc:330] T 0a295f76cdb24591992511d46af4e212 P 4b7c2656c39d4700b584200bfcc1a115: stopping tablet replica
I20260809 08:56:07.723855 7479 raft_consensus.cc:2229] T 0a295f76cdb24591992511d46af4e212 P 4b7c2656c39d4700b584200bfcc1a115 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:56:07.723874 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 6793
I20260809 08:56:07.733170 7310 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a295f76cdb24591992511d46af4e212 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:56:07 UTC) from {username='kudu', principal='kudu/127.8.154.62@KRBTEST.COM'} at 127.0.0.1:56926
I20260809 08:56:07.733394 7479 raft_consensus.cc:2258] T 0a295f76cdb24591992511d46af4e212 P 4b7c2656c39d4700b584200bfcc1a115 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:56:07.733461 7482 tablet_replica.cc:330] T 0a295f76cdb24591992511d46af4e212 P d1a1c90b5feb493fa0baca4b889ee07e: stopping tablet replica
I20260809 08:56:07.733597 7482 raft_consensus.cc:2229] T 0a295f76cdb24591992511d46af4e212 P d1a1c90b5feb493fa0baca4b889ee07e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:56:07.733734 7482 raft_consensus.cc:2258] T 0a295f76cdb24591992511d46af4e212 P d1a1c90b5feb493fa0baca4b889ee07e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:56:07.734007 7479 ts_tablet_manager.cc:1905] T 0a295f76cdb24591992511d46af4e212 P 4b7c2656c39d4700b584200bfcc1a115: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:56:07.734231 7482 ts_tablet_manager.cc:1905] T 0a295f76cdb24591992511d46af4e212 P d1a1c90b5feb493fa0baca4b889ee07e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:56:07.735888 7482 ts_tablet_manager.cc:1918] T 0a295f76cdb24591992511d46af4e212 P d1a1c90b5feb493fa0baca4b889ee07e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:56:07.735980 7482 log.cc:1196] T 0a295f76cdb24591992511d46af4e212 P d1a1c90b5feb493fa0baca4b889ee07e: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-2/wal/wals/0a295f76cdb24591992511d46af4e212
I20260809 08:56:07.736112 7479 ts_tablet_manager.cc:1918] T 0a295f76cdb24591992511d46af4e212 P 4b7c2656c39d4700b584200bfcc1a115: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:56:07.736212 7479 log.cc:1196] T 0a295f76cdb24591992511d46af4e212 P 4b7c2656c39d4700b584200bfcc1a115: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265519942966-8808-0/minicluster-data/ts-1/wal/wals/0a295f76cdb24591992511d46af4e212
I20260809 08:56:07.736343 7482 ts_tablet_manager.cc:1939] T 0a295f76cdb24591992511d46af4e212 P d1a1c90b5feb493fa0baca4b889ee07e: Deleting consensus metadata
I20260809 08:56:07.736543 7479 ts_tablet_manager.cc:1939] T 0a295f76cdb24591992511d46af4e212 P 4b7c2656c39d4700b584200bfcc1a115: Deleting consensus metadata
I20260809 08:56:07.736878 6669 catalog_manager.cc:4841] TS d1a1c90b5feb493fa0baca4b889ee07e (127.8.154.3:37399): tablet 0a295f76cdb24591992511d46af4e212 (table test-table [id=fba5146be7c4446ead9b046a6c01c147]) successfully deleted
I20260809 08:56:07.737001 6669 catalog_manager.cc:4841] TS 4b7c2656c39d4700b584200bfcc1a115 (127.8.154.2:44053): tablet 0a295f76cdb24591992511d46af4e212 (table test-table [id=fba5146be7c4446ead9b046a6c01c147]) successfully deleted
W20260809 08:56:07.738848 6669 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.154.1:34095 (error 108)
I20260809 08:56:07.739163 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 6997
I20260809 08:56:07.739393 7436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:07.739264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:34095 (local address 127.0.0.1:43096)
0809 08:56:07.739312 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:07.739353 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.1:34095: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260809 08:56:07.745116 6669 catalog_manager.cc:4584] TS 69d32892acbb4fc4b55e60397bf6e16e (127.8.154.1:34095): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0a295f76cdb24591992511d46af4e212: Network error: Client connection negotiation failed: client connection to 127.8.154.1:34095: connect: Connection refused (error 111)
I20260809 08:56:07.748478 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 7208
I20260809 08:56:07.763686 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 6644
2026-08-09T08:56:07Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2306 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T08:56:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:56:07Z Disabled control of system clock
I20260809 08:56:07.799273 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:35349
--webserver_interface=127.8.154.62
--webserver_port=0
--builtin_ntp_servers=127.8.154.20:43655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:35349
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 08:56:07.881251 7495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:07.881456 7495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:07.881476 7495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:07.882550 7495 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:56:07.882586 7495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:07.882598 7495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:07.882611 7495 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:56:07.882622 7495 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:56:07.883976 7495 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:43655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:35349
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:35349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:07.884236 7495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:07.884487 7495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:07.889345 7503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:07.889330 7507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:07.889330 7500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:07.889590 7495 server_base.cc:1034] running on GCE node
I20260809 08:56:07.889926 7495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:07.890275 7495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:07.891451 7495 hybrid_clock.cc:648] HybridClock initialized: now 1786265767891447 us; error 39 us; skew 500 ppm
I20260809 08:56:07.892795 7495 webserver.cc:460] Webserver started at http://127.8.154.62:40981/ using document root <none> and password file <none>
I20260809 08:56:07.893074 7495 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:07.893148 7495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:07.893308 7495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:07.894446 7495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "f47a2ce6d0814bf49359f12d75c8ac5e"
format_stamp: "Formatted at 2026-08-09 08:56:07 on dist-test-slave-t3q3"
server_key: "512dd7d1c73bd75c20b3dea0d94a5836"
server_key_iv: "47f9fe2219df094623486aea3f17a3e6"
server_key_version: "encryptionkey@0"
I20260809 08:56:07.894799 7495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "f47a2ce6d0814bf49359f12d75c8ac5e"
format_stamp: "Formatted at 2026-08-09 08:56:07 on dist-test-slave-t3q3"
server_key: "512dd7d1c73bd75c20b3dea0d94a5836"
server_key_iv: "47f9fe2219df094623486aea3f17a3e6"
server_key_version: "encryptionkey@0"
I20260809 08:56:07.896157 7495 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:56:07.896911 7518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:07.897126 7495 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:56:07.897234 7495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "f47a2ce6d0814bf49359f12d75c8ac5e"
format_stamp: "Formatted at 2026-08-09 08:56:07 on dist-test-slave-t3q3"
server_key: "512dd7d1c73bd75c20b3dea0d94a5836"
server_key_iv: "47f9fe2219df094623486aea3f17a3e6"
server_key_version: "encryptionkey@0"
I20260809 08:56:07.897308 7495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:07.909641 7495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:07.909932 7495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:07.910089 7495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:07.913976 7495 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:35349
I20260809 08:56:07.914044 7593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:35349 every 8 connection(s)
I20260809 08:56:07.914374 7495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:56:07.915403 7594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:07.917816 7594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f47a2ce6d0814bf49359f12d75c8ac5e: Bootstrap starting.
I20260809 08:56:07.918414 7594 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f47a2ce6d0814bf49359f12d75c8ac5e: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:07.918725 7594 log.cc:824] T 00000000000000000000000000000000 P f47a2ce6d0814bf49359f12d75c8ac5e: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:07.919430 7594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f47a2ce6d0814bf49359f12d75c8ac5e: No bootstrap required, opened a new log
I20260809 08:56:07.921132 7594 raft_consensus.cc:348] T 00000000000000000000000000000000 P f47a2ce6d0814bf49359f12d75c8ac5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f47a2ce6d0814bf49359f12d75c8ac5e" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 35349 } }
I20260809 08:56:07.921274 7594 raft_consensus.cc:374] T 00000000000000000000000000000000 P f47a2ce6d0814bf49359f12d75c8ac5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:07.921298 7594 raft_consensus.cc:729] T 00000000000000000000000000000000 P f47a2ce6d0814bf49359f12d75c8ac5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f47a2ce6d0814bf49359f12d75c8ac5e, State: Initialized, Role: FOLLOWER
I20260809 08:56:07.921509 7594 consensus_queue.cc:260] T 00000000000000000000000000000000 P f47a2ce6d0814bf49359f12d75c8ac5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f47a2ce6d0814bf49359f12d75c8ac5e" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 35349 } }
I20260809 08:56:07.921581 7594 raft_consensus.cc:388] T 00000000000000000000000000000000 P f47a2ce6d0814bf49359f12d75c8ac5e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:56:07.921604 7594 raft_consensus.cc:482] T 00000000000000000000000000000000 P f47a2ce6d0814bf49359f12d75c8ac5e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:56:07.921654 7594 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f47a2ce6d0814bf49359f12d75c8ac5e [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:07.922211 7594 raft_consensus.cc:504] T 00000000000000000000000000000000 P f47a2ce6d0814bf49359f12d75c8ac5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f47a2ce6d0814bf49359f12d75c8ac5e" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 35349 } }
I20260809 08:56:07.922302 7594 leader_election.cc:302] T 00000000000000000000000000000000 P f47a2ce6d0814bf49359f12d75c8ac5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f47a2ce6d0814bf49359f12d75c8ac5e; no voters:
I20260809 08:56:07.922497 7594 leader_election.cc:288] T 00000000000000000000000000000000 P f47a2ce6d0814bf49359f12d75c8ac5e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:56:07.922646 7597 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f47a2ce6d0814bf49359f12d75c8ac5e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:56:07.922793 7594 sys_catalog.cc:564] T 00000000000000000000000000000000 P f47a2ce6d0814bf49359f12d75c8ac5e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:56:07.922820 7597 raft_consensus.cc:686] T 00000000000000000000000000000000 P f47a2ce6d0814bf49359f12d75c8ac5e [term 1 LEADER]: Becoming Leader. State: Replica: f47a2ce6d0814bf49359f12d75c8ac5e, State: Running, Role: LEADER
I20260809 08:56:07.922920 7597 consensus_queue.cc:237] T 00000000000000000000000000000000 P f47a2ce6d0814bf49359f12d75c8ac5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f47a2ce6d0814bf49359f12d75c8ac5e" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 35349 } }
I20260809 08:56:07.923323 7598 sys_catalog.cc:455] T 00000000000000000000000000000000 P f47a2ce6d0814bf49359f12d75c8ac5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f47a2ce6d0814bf49359f12d75c8ac5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f47a2ce6d0814bf49359f12d75c8ac5e" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 35349 } } }
I20260809 08:56:07.923425 7598 sys_catalog.cc:458] T 00000000000000000000000000000000 P f47a2ce6d0814bf49359f12d75c8ac5e [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:07.923414 7602 sys_catalog.cc:455] T 00000000000000000000000000000000 P f47a2ce6d0814bf49359f12d75c8ac5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f47a2ce6d0814bf49359f12d75c8ac5e. Latest consensus state: current_term: 1 leader_uuid: "f47a2ce6d0814bf49359f12d75c8ac5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f47a2ce6d0814bf49359f12d75c8ac5e" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 35349 } } }
I20260809 08:56:07.923569 7602 sys_catalog.cc:458] T 00000000000000000000000000000000 P f47a2ce6d0814bf49359f12d75c8ac5e [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:07.923766 7615 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:56:07.923877 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 7495
I20260809 08:56:07.923969 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:56:07.924297 8808 external_mini_cluster.cc:1428] Setting key 7b07fdfbed11fd760a99f48af360721c
I20260809 08:56:07.924360 7615 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:56:07.926028 7615 catalog_manager.cc:1347] Generated new cluster ID: 0e6907944b064a14a2b2f5435c913f4f
I20260809 08:56:07.926088 7615 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:56:07.930549 7629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:07.924790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33320 (local address 127.8.154.62:35349)
0809 08:56:07.925049 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:07.925058 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:07.925094 (+ 36us) server_negotiation.cc:407] Connection header received
0809 08:56:07.925316 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:07.925323 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:07.925397 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:07.925560 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:07.925805 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:07.927177 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:07.928271 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:07.928951 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:07.929096 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:07.929792 (+ 696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:07.929797 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:07.929797 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:56:07.930018 (+ 221us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:07.930068 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:56:07.930083 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:07.930157 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:07.930231 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 08:56:07.930312 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":74,"threads_started":1}
I20260809 08:56:07.942408 7615 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:56:07.942943 7615 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:56:07.947229 7615 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f47a2ce6d0814bf49359f12d75c8ac5e: Generated new TSK 0
I20260809 08:56:07.947435 7615 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:56:07.950169 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.1:0
--local_ip_for_outbound_sockets=127.8.154.1
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:35349
--builtin_ntp_servers=127.8.154.20:43655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 08:56:08.034581 7633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:08.034781 7633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:08.034802 7633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:08.035989 7633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:08.036036 7633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:08.036115 7633 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.1
I20260809 08:56:08.037462 7633 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:43655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.8.154.1
--webserver_port=0
--tserver_master_addrs=127.8.154.62:35349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.1
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:08.037710 7633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:08.038030 7633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:08.044225 7651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:08.044199 7645 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:08.044428 7633 server_base.cc:1034] running on GCE node
W20260809 08:56:08.044209 7644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:08.044747 7633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:08.044986 7633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:08.046168 7633 hybrid_clock.cc:648] HybridClock initialized: now 1786265768046152 us; error 29 us; skew 500 ppm
I20260809 08:56:08.047456 7633 webserver.cc:460] Webserver started at http://127.8.154.1:40807/ using document root <none> and password file <none>
I20260809 08:56:08.047727 7633 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:08.047802 7633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:08.047956 7633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:08.049050 7633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-0/data/instance:
uuid: "e1fbdfb76fed4aae80b18ce233b2c658"
format_stamp: "Formatted at 2026-08-09 08:56:08 on dist-test-slave-t3q3"
server_key: "4fb264242fe3513ea8818df48c754b6a"
server_key_iv: "544d3d089c83fb66f50874ce833ef2aa"
server_key_version: "encryptionkey@0"
I20260809 08:56:08.049410 7633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance:
uuid: "e1fbdfb76fed4aae80b18ce233b2c658"
format_stamp: "Formatted at 2026-08-09 08:56:08 on dist-test-slave-t3q3"
server_key: "4fb264242fe3513ea8818df48c754b6a"
server_key_iv: "544d3d089c83fb66f50874ce833ef2aa"
server_key_version: "encryptionkey@0"
I20260809 08:56:08.050709 7633 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:56:08.051415 7662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:08.051590 7633 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:56:08.051692 7633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-0/wal
uuid: "e1fbdfb76fed4aae80b18ce233b2c658"
format_stamp: "Formatted at 2026-08-09 08:56:08 on dist-test-slave-t3q3"
server_key: "4fb264242fe3513ea8818df48c754b6a"
server_key_iv: "544d3d089c83fb66f50874ce833ef2aa"
server_key_version: "encryptionkey@0"
I20260809 08:56:08.051788 7633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:08.071142 7633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:08.071424 7633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:08.071573 7633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:08.071796 7633 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:56:08.072194 7633 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:56:08.072233 7633 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:08.072254 7633 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:56:08.072296 7633 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:08.078069 7633 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.1:33455
I20260809 08:56:08.078167 7847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.1:33455 every 8 connection(s)
I20260809 08:56:08.078469 7633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-0/data/info.pb
I20260809 08:56:08.083956 7629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:08.079228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:32885 (local address 127.8.154.62:35349)
0809 08:56:08.079291 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:08.079309 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:08.079530 (+ 221us) server_negotiation.cc:407] Connection header received
0809 08:56:08.080124 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:08.080126 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:08.080143 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:08.080193 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:08.080909 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:08.081427 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:08.082467 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:08.082681 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:08.082763 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:08.083641 (+ 878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:08.083643 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:08.083643 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:56:08.083651 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:08.083672 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:56:08.083676 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:08.083731 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:08.083844 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 08:56:08.083876 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 08:56:08.084213 7852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:08.079199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:35349 (local address 127.8.154.1:32885)
0809 08:56:08.079406 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:08.079439 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:08.079953 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:08.080276 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:08.080280 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:08.080291 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:56:08.080802 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:08.080808 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:08.081536 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:08.081537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:08.082364 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:08.082366 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:08.082772 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:08.082773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:08.082921 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:08.083484 (+ 563us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:56:08.083484 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:56:08.083514 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:56:08.083537 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:56:08.083744 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:56:08.083751 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:56:08.083755 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:56:08.083844 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 08:56:08.083950 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":82,"threads_started":1}
I20260809 08:56:08.084780 7848 heartbeater.cc:347] Connected to a master server at 127.8.154.62:35349
I20260809 08:56:08.084877 7848 heartbeater.cc:464] Registering TS with master...
I20260809 08:56:08.085028 7848 heartbeater.cc:511] Master 127.8.154.62:35349 requested a full tablet report, sending...
I20260809 08:56:08.085548 7541 ts_manager.cc:194] Registered new tserver with Master: e1fbdfb76fed4aae80b18ce233b2c658 (127.8.154.1:33455)
I20260809 08:56:08.086238 7541 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.154.1:32885
I20260809 08:56:08.086395 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 7633
I20260809 08:56:08.086509 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-0/wal/instance
I20260809 08:56:08.086726 8808 external_mini_cluster.cc:1428] Setting key 65984e0e05c97b1482aba7dea65f6140
I20260809 08:56:08.087651 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.2:0
--local_ip_for_outbound_sockets=127.8.154.2
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:35349
--builtin_ntp_servers=127.8.154.20:43655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 08:56:08.170785 7854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:08.170979 7854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:08.171002 7854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:08.172158 7854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:08.172199 7854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:08.172222 7854 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.2
I20260809 08:56:08.173552 7854 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:43655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.8.154.2
--webserver_port=0
--tserver_master_addrs=127.8.154.62:35349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7854
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.2
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:08.173801 7854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:08.174103 7854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:08.179064 7861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:08.179171 7860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:08.179077 7864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:08.179510 7854 server_base.cc:1034] running on GCE node
I20260809 08:56:08.179745 7854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:08.179981 7854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:08.181183 7854 hybrid_clock.cc:648] HybridClock initialized: now 1786265768181170 us; error 27 us; skew 500 ppm
I20260809 08:56:08.182545 7854 webserver.cc:460] Webserver started at http://127.8.154.2:46875/ using document root <none> and password file <none>
I20260809 08:56:08.182787 7854 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:08.182857 7854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:08.183013 7854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:08.184113 7854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-1/data/instance:
uuid: "c8ee50a8a25644de84f69451956e709d"
format_stamp: "Formatted at 2026-08-09 08:56:08 on dist-test-slave-t3q3"
server_key: "b38c81645edd77ed11aa8de93cf9b248"
server_key_iv: "01ab684ca2b515f7416950618c435bf1"
server_key_version: "encryptionkey@0"
I20260809 08:56:08.184514 7854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance:
uuid: "c8ee50a8a25644de84f69451956e709d"
format_stamp: "Formatted at 2026-08-09 08:56:08 on dist-test-slave-t3q3"
server_key: "b38c81645edd77ed11aa8de93cf9b248"
server_key_iv: "01ab684ca2b515f7416950618c435bf1"
server_key_version: "encryptionkey@0"
I20260809 08:56:08.185926 7854 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:56:08.186749 7872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:08.186949 7854 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:56:08.187059 7854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-1/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-1/wal
uuid: "c8ee50a8a25644de84f69451956e709d"
format_stamp: "Formatted at 2026-08-09 08:56:08 on dist-test-slave-t3q3"
server_key: "b38c81645edd77ed11aa8de93cf9b248"
server_key_iv: "01ab684ca2b515f7416950618c435bf1"
server_key_version: "encryptionkey@0"
I20260809 08:56:08.187147 7854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:08.213333 7854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:08.213671 7854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:08.213832 7854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:08.214087 7854 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:56:08.214440 7854 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:56:08.214496 7854 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:08.214548 7854 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:56:08.214596 7854 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:08.220484 7854 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.2:46809
I20260809 08:56:08.220534 8052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.2:46809 every 8 connection(s)
I20260809 08:56:08.220901 7854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-1/data/info.pb
I20260809 08:56:08.223361 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 7854
I20260809 08:56:08.223487 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-1/wal/instance
I20260809 08:56:08.223726 8808 external_mini_cluster.cc:1428] Setting key 99a6ab4e74f75dc73b80a7c316d39862
I20260809 08:56:08.224771 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.8.154.3:0
--local_ip_for_outbound_sockets=127.8.154.3
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:35349
--builtin_ntp_servers=127.8.154.20:43655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 08:56:08.227072 7629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:08.221650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.2:46453 (local address 127.8.154.62:35349)
0809 08:56:08.221696 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:08.221699 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:08.221937 (+ 238us) server_negotiation.cc:407] Connection header received
0809 08:56:08.222524 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:08.222525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:08.222536 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:08.222599 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:08.223541 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:08.224277 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:08.225465 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:08.225777 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:08.225857 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:08.226763 (+ 906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:08.226764 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:08.226765 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:56:08.226770 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:08.226788 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:56:08.226793 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:08.226855 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:08.226936 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 08:56:08.226966 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 08:56:08.227231 8055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:08.221612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:35349 (local address 127.8.154.2:46453)
0809 08:56:08.221804 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:08.221846 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:08.222422 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:08.222683 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:08.222686 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:08.222695 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:56:08.223431 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:08.223436 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:08.224390 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:08.224391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:08.225375 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:08.225378 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:08.225869 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:08.225869 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:08.225982 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:08.226613 (+ 631us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:56:08.226614 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:56:08.226643 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:56:08.226667 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:56:08.226862 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:56:08.226867 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:56:08.226869 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 08:56:08.226922 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 08:56:08.227001 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
I20260809 08:56:08.227831 8053 heartbeater.cc:347] Connected to a master server at 127.8.154.62:35349
I20260809 08:56:08.227945 8053 heartbeater.cc:464] Registering TS with master...
I20260809 08:56:08.228185 8053 heartbeater.cc:511] Master 127.8.154.62:35349 requested a full tablet report, sending...
I20260809 08:56:08.228603 7541 ts_manager.cc:194] Registered new tserver with Master: c8ee50a8a25644de84f69451956e709d (127.8.154.2:46809)
I20260809 08:56:08.228981 7541 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.154.2:46453
W20260809 08:56:08.308550 8057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:08.308758 8057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:08.308779 8057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:08.309902 8057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:08.309942 8057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:08.309966 8057 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.8.154.3
I20260809 08:56:08.311378 8057 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:43655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.8.154.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.8.154.3
--webserver_port=0
--tserver_master_addrs=127.8.154.62:35349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8057
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.8.154.3
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:08.311638 8057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:08.311931 8057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:08.317106 8063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:08.317097 8062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:08.317466 8057 server_base.cc:1034] running on GCE node
W20260809 08:56:08.317166 8065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:08.317730 8057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:08.317926 8057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:08.319062 8057 hybrid_clock.cc:648] HybridClock initialized: now 1786265768319064 us; error 43 us; skew 500 ppm
I20260809 08:56:08.320338 8057 webserver.cc:460] Webserver started at http://127.8.154.3:33171/ using document root <none> and password file <none>
I20260809 08:56:08.320571 8057 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:08.320626 8057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:08.320770 8057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:08.321811 8057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-2/data/instance:
uuid: "3c2587539cec4459b362f77d0448ccd8"
format_stamp: "Formatted at 2026-08-09 08:56:08 on dist-test-slave-t3q3"
server_key: "2daf1c499b7ee7ff1c8e2741c4926dbd"
server_key_iv: "54a4894ac35844d87ecb717da22eb324"
server_key_version: "encryptionkey@0"
I20260809 08:56:08.322196 8057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance:
uuid: "3c2587539cec4459b362f77d0448ccd8"
format_stamp: "Formatted at 2026-08-09 08:56:08 on dist-test-slave-t3q3"
server_key: "2daf1c499b7ee7ff1c8e2741c4926dbd"
server_key_iv: "54a4894ac35844d87ecb717da22eb324"
server_key_version: "encryptionkey@0"
I20260809 08:56:08.323592 8057 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:56:08.324523 8075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:08.324699 8057 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:56:08.324803 8057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-2/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-2/wal
uuid: "3c2587539cec4459b362f77d0448ccd8"
format_stamp: "Formatted at 2026-08-09 08:56:08 on dist-test-slave-t3q3"
server_key: "2daf1c499b7ee7ff1c8e2741c4926dbd"
server_key_iv: "54a4894ac35844d87ecb717da22eb324"
server_key_version: "encryptionkey@0"
I20260809 08:56:08.324887 8057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:08.332674 8057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:08.332930 8057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:08.333087 8057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:08.333310 8057 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:56:08.333626 8057 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:56:08.333679 8057 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:08.333720 8057 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:56:08.333750 8057 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:08.339576 8057 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.3:45013
I20260809 08:56:08.339632 8263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.3:45013 every 8 connection(s)
I20260809 08:56:08.339934 8057 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-2/data/info.pb
I20260809 08:56:08.340919 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 8057
I20260809 08:56:08.341048 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-2/wal/instance
I20260809 08:56:08.341259 8808 external_mini_cluster.cc:1428] Setting key 07853663b154cdd536a40d6beeb84797
I20260809 08:56:08.345486 7629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:08.340654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.3:44213 (local address 127.8.154.62:35349)
0809 08:56:08.340715 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:08.340717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:08.341014 (+ 297us) server_negotiation.cc:407] Connection header received
0809 08:56:08.341670 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:08.341671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:08.341685 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:08.341736 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:08.342400 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:08.342906 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:08.344013 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:08.344305 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:08.344382 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:08.345180 (+ 798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:08.345181 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:08.345182 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:56:08.345186 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:08.345202 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:56:08.345206 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:08.345279 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:08.345374 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 08:56:08.345408 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 08:56:08.345753 8268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:08.340634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:35349 (local address 127.8.154.3:44213)
0809 08:56:08.340861 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:08.340907 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:08.341551 (+ 644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:08.341800 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:08.341804 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:08.341814 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:56:08.342307 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:08.342311 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:08.343001 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:08.343002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:08.343798 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:08.343801 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:08.344407 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:08.344408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:08.344545 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:08.344998 (+ 453us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:56:08.344998 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:56:08.345026 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:56:08.345048 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:56:08.345280 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:56:08.345287 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:56:08.345290 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:56:08.345376 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 08:56:08.345476 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":114,"threads_started":1}
I20260809 08:56:08.346212 8264 heartbeater.cc:347] Connected to a master server at 127.8.154.62:35349
I20260809 08:56:08.346288 8264 heartbeater.cc:464] Registering TS with master...
I20260809 08:56:08.346473 8264 heartbeater.cc:511] Master 127.8.154.62:35349 requested a full tablet report, sending...
I20260809 08:56:08.346870 7541 ts_manager.cc:194] Registered new tserver with Master: 3c2587539cec4459b362f77d0448ccd8 (127.8.154.3:45013)
I20260809 08:56:08.347261 7541 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.8.154.3:44213
I20260809 08:56:08.353024 8808 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:56:08.357311 7629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:08.354481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33324 (local address 127.8.154.62:35349)
0809 08:56:08.354542 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:08.354544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:08.354659 (+ 115us) server_negotiation.cc:407] Connection header received
0809 08:56:08.354712 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:08.354713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:08.354723 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:08.354778 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:08.354929 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:08.355512 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:08.356282 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:08.356497 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:08.356558 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:08.357122 (+ 564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:08.357123 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:08.357124 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:56:08.357128 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:08.357140 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:56:08.357144 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:08.357189 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:08.357233 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 08:56:08.357254 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 08:56:08.358825 7541 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33324:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:56:08.360430 7541 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:56:08.366319 8277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:08.362528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:46809 (local address 127.0.0.1:41138)
0809 08:56:08.362645 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:08.362681 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:08.362741 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:08.363173 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:08.363175 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:08.363177 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:56:08.363251 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:08.363252 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:08.364280 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:08.364281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:08.365067 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:08.365069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:08.366033 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:08.366034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:08.366110 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:08.366124 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 08:56:08.366197 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 08:56:08.366249 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":75,"threads_started":1}
I20260809 08:56:08.366336 8283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:08.362809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41138 (local address 127.8.154.2:46809)
0809 08:56:08.362986 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:08.362990 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:08.363003 (+ 13us) server_negotiation.cc:407] Connection header received
0809 08:56:08.363055 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:08.363057 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:08.363073 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:08.363161 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:56:08.363322 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:08.364197 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:08.365214 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:08.365966 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:08.366033 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:08.366132 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:08.366211 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 08:56:08.366249 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":103,"threads_started":1}
I20260809 08:56:08.367287 7945 tablet_service.cc:1467] Processing CreateTablet for tablet e402d0a40bee47c5a1c18079e0f7ef9e (DEFAULT_TABLE table=test-table [id=2ddaf8d442ed46f1842c3d66250c1383]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:56:08.367660 7945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e402d0a40bee47c5a1c18079e0f7ef9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:08.368908 8280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:08.362532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:45013 (local address 127.0.0.1:44992)
0809 08:56:08.365310 (+ 2778us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:08.365320 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:08.365402 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:08.365602 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:08.365603 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:08.365605 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:56:08.365643 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:08.365645 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:08.366914 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:08.366915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:08.367863 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:08.367866 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:08.368663 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:08.368664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:08.368730 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:08.368734 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:56:08.368797 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 08:56:08.368836 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2749,"mutex_wait_us":22,"thread_start_us":92,"threads_started":1}
I20260809 08:56:08.368927 8282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:08.362763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44992 (local address 127.8.154.3:45013)
0809 08:56:08.362915 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:08.362919 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:08.365395 (+ 2476us) server_negotiation.cc:407] Connection header received
0809 08:56:08.365472 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:08.365475 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:08.365491 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:08.365584 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:56:08.365714 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:08.366820 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:08.367949 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:08.368572 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:08.368661 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:08.368746 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:08.368813 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 08:56:08.368847 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":86,"threads_started":1}
I20260809 08:56:08.369405 8278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:08.362562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:33455 (local address 127.0.0.1:60416)
0809 08:56:08.365586 (+ 3024us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:08.365596 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:08.365643 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:08.365847 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:08.365848 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:08.365850 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:56:08.365882 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:08.365884 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:08.367064 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:08.367065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:08.368225 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:08.368228 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:08.369141 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:08.369142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:08.369212 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:08.369217 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:56:08.369290 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 08:56:08.369332 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3010,"thread_start_us":96,"threads_started":1}
I20260809 08:56:08.369451 8285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:08.364220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60416 (local address 127.8.154.1:33455)
0809 08:56:08.365532 (+ 1312us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:08.365536 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:08.365695 (+ 159us) server_negotiation.cc:407] Connection header received
0809 08:56:08.365739 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:08.365741 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:08.365758 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:08.365835 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:56:08.365944 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:08.366988 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:08.368328 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:08.369046 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:08.369137 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:08.369220 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:08.369303 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 08:56:08.369344 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1270,"thread_start_us":975,"threads_started":1}
I20260809 08:56:08.369841 8152 tablet_service.cc:1467] Processing CreateTablet for tablet e402d0a40bee47c5a1c18079e0f7ef9e (DEFAULT_TABLE table=test-table [id=2ddaf8d442ed46f1842c3d66250c1383]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:56:08.370164 8152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e402d0a40bee47c5a1c18079e0f7ef9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:08.370086 7743 tablet_service.cc:1467] Processing CreateTablet for tablet e402d0a40bee47c5a1c18079e0f7ef9e (DEFAULT_TABLE table=test-table [id=2ddaf8d442ed46f1842c3d66250c1383]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:56:08.370425 7743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e402d0a40bee47c5a1c18079e0f7ef9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:08.370903 8286 tablet_bootstrap.cc:492] T e402d0a40bee47c5a1c18079e0f7ef9e P c8ee50a8a25644de84f69451956e709d: Bootstrap starting.
I20260809 08:56:08.371482 8286 tablet_bootstrap.cc:654] T e402d0a40bee47c5a1c18079e0f7ef9e P c8ee50a8a25644de84f69451956e709d: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:08.371711 8286 log.cc:824] T e402d0a40bee47c5a1c18079e0f7ef9e P c8ee50a8a25644de84f69451956e709d: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:08.372371 8286 tablet_bootstrap.cc:492] T e402d0a40bee47c5a1c18079e0f7ef9e P c8ee50a8a25644de84f69451956e709d: No bootstrap required, opened a new log
I20260809 08:56:08.372416 8286 ts_tablet_manager.cc:1397] T e402d0a40bee47c5a1c18079e0f7ef9e P c8ee50a8a25644de84f69451956e709d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:56:08.373543 8288 tablet_bootstrap.cc:492] T e402d0a40bee47c5a1c18079e0f7ef9e P 3c2587539cec4459b362f77d0448ccd8: Bootstrap starting.
I20260809 08:56:08.373826 8289 tablet_bootstrap.cc:492] T e402d0a40bee47c5a1c18079e0f7ef9e P e1fbdfb76fed4aae80b18ce233b2c658: Bootstrap starting.
I20260809 08:56:08.374202 8288 tablet_bootstrap.cc:654] T e402d0a40bee47c5a1c18079e0f7ef9e P 3c2587539cec4459b362f77d0448ccd8: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:08.374469 8289 tablet_bootstrap.cc:654] T e402d0a40bee47c5a1c18079e0f7ef9e P e1fbdfb76fed4aae80b18ce233b2c658: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:08.374522 8288 log.cc:824] T e402d0a40bee47c5a1c18079e0f7ef9e P 3c2587539cec4459b362f77d0448ccd8: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:08.374828 8289 log.cc:824] T e402d0a40bee47c5a1c18079e0f7ef9e P e1fbdfb76fed4aae80b18ce233b2c658: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:08.375224 8288 tablet_bootstrap.cc:492] T e402d0a40bee47c5a1c18079e0f7ef9e P 3c2587539cec4459b362f77d0448ccd8: No bootstrap required, opened a new log
I20260809 08:56:08.375092 8286 raft_consensus.cc:348] T e402d0a40bee47c5a1c18079e0f7ef9e P c8ee50a8a25644de84f69451956e709d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c2587539cec4459b362f77d0448ccd8" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45013 } } peers { permanent_uuid: "c8ee50a8a25644de84f69451956e709d" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 46809 } } peers { permanent_uuid: "e1fbdfb76fed4aae80b18ce233b2c658" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 33455 } }
I20260809 08:56:08.375265 8286 raft_consensus.cc:374] T e402d0a40bee47c5a1c18079e0f7ef9e P c8ee50a8a25644de84f69451956e709d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:08.375280 8288 ts_tablet_manager.cc:1397] T e402d0a40bee47c5a1c18079e0f7ef9e P 3c2587539cec4459b362f77d0448ccd8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:56:08.375319 8286 raft_consensus.cc:729] T e402d0a40bee47c5a1c18079e0f7ef9e P c8ee50a8a25644de84f69451956e709d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8ee50a8a25644de84f69451956e709d, State: Initialized, Role: FOLLOWER
I20260809 08:56:08.375452 8286 consensus_queue.cc:260] T e402d0a40bee47c5a1c18079e0f7ef9e P c8ee50a8a25644de84f69451956e709d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c2587539cec4459b362f77d0448ccd8" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45013 } } peers { permanent_uuid: "c8ee50a8a25644de84f69451956e709d" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 46809 } } peers { permanent_uuid: "e1fbdfb76fed4aae80b18ce233b2c658" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 33455 } }
I20260809 08:56:08.375744 8289 tablet_bootstrap.cc:492] T e402d0a40bee47c5a1c18079e0f7ef9e P e1fbdfb76fed4aae80b18ce233b2c658: No bootstrap required, opened a new log
I20260809 08:56:08.375739 8286 ts_tablet_manager.cc:1428] T e402d0a40bee47c5a1c18079e0f7ef9e P c8ee50a8a25644de84f69451956e709d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:56:08.375830 8289 ts_tablet_manager.cc:1397] T e402d0a40bee47c5a1c18079e0f7ef9e P e1fbdfb76fed4aae80b18ce233b2c658: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:56:08.375856 8053 heartbeater.cc:503] Master 127.8.154.62:35349 was elected leader, sending a full tablet report...
I20260809 08:56:08.377736 8288 raft_consensus.cc:348] T e402d0a40bee47c5a1c18079e0f7ef9e P 3c2587539cec4459b362f77d0448ccd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c2587539cec4459b362f77d0448ccd8" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45013 } } peers { permanent_uuid: "c8ee50a8a25644de84f69451956e709d" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 46809 } } peers { permanent_uuid: "e1fbdfb76fed4aae80b18ce233b2c658" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 33455 } }
I20260809 08:56:08.377861 8288 raft_consensus.cc:374] T e402d0a40bee47c5a1c18079e0f7ef9e P 3c2587539cec4459b362f77d0448ccd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:08.377882 8288 raft_consensus.cc:729] T e402d0a40bee47c5a1c18079e0f7ef9e P 3c2587539cec4459b362f77d0448ccd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c2587539cec4459b362f77d0448ccd8, State: Initialized, Role: FOLLOWER
I20260809 08:56:08.378060 8288 consensus_queue.cc:260] T e402d0a40bee47c5a1c18079e0f7ef9e P 3c2587539cec4459b362f77d0448ccd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c2587539cec4459b362f77d0448ccd8" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45013 } } peers { permanent_uuid: "c8ee50a8a25644de84f69451956e709d" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 46809 } } peers { permanent_uuid: "e1fbdfb76fed4aae80b18ce233b2c658" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 33455 } }
I20260809 08:56:08.378347 8288 ts_tablet_manager.cc:1428] T e402d0a40bee47c5a1c18079e0f7ef9e P 3c2587539cec4459b362f77d0448ccd8: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 08:56:08.378264 8289 raft_consensus.cc:348] T e402d0a40bee47c5a1c18079e0f7ef9e P e1fbdfb76fed4aae80b18ce233b2c658 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c2587539cec4459b362f77d0448ccd8" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45013 } } peers { permanent_uuid: "c8ee50a8a25644de84f69451956e709d" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 46809 } } peers { permanent_uuid: "e1fbdfb76fed4aae80b18ce233b2c658" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 33455 } }
I20260809 08:56:08.378415 8289 raft_consensus.cc:374] T e402d0a40bee47c5a1c18079e0f7ef9e P e1fbdfb76fed4aae80b18ce233b2c658 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:08.378422 8264 heartbeater.cc:503] Master 127.8.154.62:35349 was elected leader, sending a full tablet report...
I20260809 08:56:08.378466 8289 raft_consensus.cc:729] T e402d0a40bee47c5a1c18079e0f7ef9e P e1fbdfb76fed4aae80b18ce233b2c658 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1fbdfb76fed4aae80b18ce233b2c658, State: Initialized, Role: FOLLOWER
I20260809 08:56:08.378562 8289 consensus_queue.cc:260] T e402d0a40bee47c5a1c18079e0f7ef9e P e1fbdfb76fed4aae80b18ce233b2c658 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c2587539cec4459b362f77d0448ccd8" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45013 } } peers { permanent_uuid: "c8ee50a8a25644de84f69451956e709d" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 46809 } } peers { permanent_uuid: "e1fbdfb76fed4aae80b18ce233b2c658" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 33455 } }
I20260809 08:56:08.378827 8289 ts_tablet_manager.cc:1428] T e402d0a40bee47c5a1c18079e0f7ef9e P e1fbdfb76fed4aae80b18ce233b2c658: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:56:08.378890 7848 heartbeater.cc:503] Master 127.8.154.62:35349 was elected leader, sending a full tablet report...
I20260809 08:56:08.424947 8296 raft_consensus.cc:482] T e402d0a40bee47c5a1c18079e0f7ef9e P e1fbdfb76fed4aae80b18ce233b2c658 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:56:08.425076 8296 raft_consensus.cc:504] T e402d0a40bee47c5a1c18079e0f7ef9e P e1fbdfb76fed4aae80b18ce233b2c658 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c2587539cec4459b362f77d0448ccd8" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45013 } } peers { permanent_uuid: "c8ee50a8a25644de84f69451956e709d" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 46809 } } peers { permanent_uuid: "e1fbdfb76fed4aae80b18ce233b2c658" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 33455 } }
I20260809 08:56:08.425410 8296 leader_election.cc:288] T e402d0a40bee47c5a1c18079e0f7ef9e P e1fbdfb76fed4aae80b18ce233b2c658 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c2587539cec4459b362f77d0448ccd8 (127.8.154.3:45013), c8ee50a8a25644de84f69451956e709d (127.8.154.2:46809)
I20260809 08:56:08.428615 8282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:08.425643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:42137 (local address 127.8.154.3:45013)
0809 08:56:08.425716 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:08.425719 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:08.425976 (+ 257us) server_negotiation.cc:407] Connection header received
0809 08:56:08.426011 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:08.426012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:08.426022 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:08.426108 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:56:08.426287 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:08.426756 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:08.427800 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:08.428182 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:08.428282 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:08.428293 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:08.428506 (+ 213us) server_negotiation.cc:299] Negotiation successful
0809 08:56:08.428553 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 08:56:08.428606 7852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:08.425585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.2:46809 (local address 127.8.154.1:52521)
0809 08:56:08.425683 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:08.425699 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:08.425750 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:08.426000 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:08.426002 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:08.426004 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:56:08.426087 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:08.426089 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:08.426776 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:08.426777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:08.427678 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:08.427681 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:08.428400 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:08.428401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:08.428471 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:08.428473 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 08:56:08.428521 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 08:56:08.428554 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 08:56:08.428613 8298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:08.425593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.3:45013 (local address 127.8.154.1:42137)
0809 08:56:08.425806 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:08.425819 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:08.425887 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:08.426117 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:08.426118 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:08.426119 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:56:08.426211 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:08.426213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:08.426836 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:08.426837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:08.427692 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:08.427695 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:08.428285 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:08.428286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:08.428398 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:08.428402 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:56:08.428460 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 08:56:08.428506 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":31,"thread_start_us":126,"threads_started":1}
I20260809 08:56:08.428794 8283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:08.425780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.8.154.1:52521 (local address 127.8.154.2:46809)
0809 08:56:08.425838 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:08.425840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:08.425850 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:56:08.425885 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:08.425886 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:08.425896 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:08.425955 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:56:08.426162 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:08.426691 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:08.427761 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:08.428315 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:08.428379 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:08.428387 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:08.428661 (+ 274us) server_negotiation.cc:299] Negotiation successful
0809 08:56:08.428710 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 08:56:08.429082 7984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e402d0a40bee47c5a1c18079e0f7ef9e" candidate_uuid: "e1fbdfb76fed4aae80b18ce233b2c658" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8ee50a8a25644de84f69451956e709d" is_pre_election: true
I20260809 08:56:08.429144 8188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e402d0a40bee47c5a1c18079e0f7ef9e" candidate_uuid: "e1fbdfb76fed4aae80b18ce233b2c658" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c2587539cec4459b362f77d0448ccd8" is_pre_election: true
I20260809 08:56:08.429234 7984 raft_consensus.cc:2454] T e402d0a40bee47c5a1c18079e0f7ef9e P c8ee50a8a25644de84f69451956e709d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1fbdfb76fed4aae80b18ce233b2c658 in term 0.
I20260809 08:56:08.429315 8188 raft_consensus.cc:2454] T e402d0a40bee47c5a1c18079e0f7ef9e P 3c2587539cec4459b362f77d0448ccd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e1fbdfb76fed4aae80b18ce233b2c658 in term 0.
I20260809 08:56:08.429445 7671 leader_election.cc:302] T e402d0a40bee47c5a1c18079e0f7ef9e P e1fbdfb76fed4aae80b18ce233b2c658 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c8ee50a8a25644de84f69451956e709d, e1fbdfb76fed4aae80b18ce233b2c658; no voters:
I20260809 08:56:08.429618 8296 raft_consensus.cc:2781] T e402d0a40bee47c5a1c18079e0f7ef9e P e1fbdfb76fed4aae80b18ce233b2c658 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:56:08.429698 8296 raft_consensus.cc:482] T e402d0a40bee47c5a1c18079e0f7ef9e P e1fbdfb76fed4aae80b18ce233b2c658 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:56:08.429723 8296 raft_consensus.cc:3037] T e402d0a40bee47c5a1c18079e0f7ef9e P e1fbdfb76fed4aae80b18ce233b2c658 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:08.430480 8296 raft_consensus.cc:504] T e402d0a40bee47c5a1c18079e0f7ef9e P e1fbdfb76fed4aae80b18ce233b2c658 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c2587539cec4459b362f77d0448ccd8" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45013 } } peers { permanent_uuid: "c8ee50a8a25644de84f69451956e709d" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 46809 } } peers { permanent_uuid: "e1fbdfb76fed4aae80b18ce233b2c658" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 33455 } }
I20260809 08:56:08.430634 8296 leader_election.cc:288] T e402d0a40bee47c5a1c18079e0f7ef9e P e1fbdfb76fed4aae80b18ce233b2c658 [CANDIDATE]: Term 1 election: Requested vote from peers 3c2587539cec4459b362f77d0448ccd8 (127.8.154.3:45013), c8ee50a8a25644de84f69451956e709d (127.8.154.2:46809)
I20260809 08:56:08.430799 8188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e402d0a40bee47c5a1c18079e0f7ef9e" candidate_uuid: "e1fbdfb76fed4aae80b18ce233b2c658" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c2587539cec4459b362f77d0448ccd8"
I20260809 08:56:08.430824 7984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e402d0a40bee47c5a1c18079e0f7ef9e" candidate_uuid: "e1fbdfb76fed4aae80b18ce233b2c658" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8ee50a8a25644de84f69451956e709d"
I20260809 08:56:08.430886 8188 raft_consensus.cc:3037] T e402d0a40bee47c5a1c18079e0f7ef9e P 3c2587539cec4459b362f77d0448ccd8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:08.430887 7984 raft_consensus.cc:3037] T e402d0a40bee47c5a1c18079e0f7ef9e P c8ee50a8a25644de84f69451956e709d [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:08.431793 8188 raft_consensus.cc:2454] T e402d0a40bee47c5a1c18079e0f7ef9e P 3c2587539cec4459b362f77d0448ccd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1fbdfb76fed4aae80b18ce233b2c658 in term 1.
I20260809 08:56:08.431793 7984 raft_consensus.cc:2454] T e402d0a40bee47c5a1c18079e0f7ef9e P c8ee50a8a25644de84f69451956e709d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e1fbdfb76fed4aae80b18ce233b2c658 in term 1.
I20260809 08:56:08.431979 7663 leader_election.cc:302] T e402d0a40bee47c5a1c18079e0f7ef9e P e1fbdfb76fed4aae80b18ce233b2c658 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c2587539cec4459b362f77d0448ccd8, e1fbdfb76fed4aae80b18ce233b2c658; no voters:
I20260809 08:56:08.432118 8296 raft_consensus.cc:2781] T e402d0a40bee47c5a1c18079e0f7ef9e P e1fbdfb76fed4aae80b18ce233b2c658 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:56:08.432277 8296 raft_consensus.cc:686] T e402d0a40bee47c5a1c18079e0f7ef9e P e1fbdfb76fed4aae80b18ce233b2c658 [term 1 LEADER]: Becoming Leader. State: Replica: e1fbdfb76fed4aae80b18ce233b2c658, State: Running, Role: LEADER
I20260809 08:56:08.432405 8296 consensus_queue.cc:237] T e402d0a40bee47c5a1c18079e0f7ef9e P e1fbdfb76fed4aae80b18ce233b2c658 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c2587539cec4459b362f77d0448ccd8" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45013 } } peers { permanent_uuid: "c8ee50a8a25644de84f69451956e709d" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 46809 } } peers { permanent_uuid: "e1fbdfb76fed4aae80b18ce233b2c658" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 33455 } }
I20260809 08:56:08.433267 7540 catalog_manager.cc:5495] T e402d0a40bee47c5a1c18079e0f7ef9e P e1fbdfb76fed4aae80b18ce233b2c658 reported cstate change: term changed from 0 to 1, leader changed from <none> to e1fbdfb76fed4aae80b18ce233b2c658 (127.8.154.1). New cstate: current_term: 1 leader_uuid: "e1fbdfb76fed4aae80b18ce233b2c658" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c2587539cec4459b362f77d0448ccd8" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c8ee50a8a25644de84f69451956e709d" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 46809 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e1fbdfb76fed4aae80b18ce233b2c658" member_type: VOTER last_known_addr { host: "127.8.154.1" port: 33455 } health_report { overall_health: HEALTHY } } }
I20260809 08:56:08.438714 8285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:08.436450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60424 (local address 127.8.154.1:33455)
0809 08:56:08.436512 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:08.436515 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:08.436523 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:56:08.436555 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:08.436560 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:08.436576 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:08.436659 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:56:08.436765 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:08.437207 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:08.437933 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:08.438139 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:08.438215 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:08.438387 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:56:08.438483 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:56:08.438576 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:08.438602 (+ 26us) server_negotiation.cc:299] Negotiation successful
0809 08:56:08.438620 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 08:56:08.440451 8188 raft_consensus.cc:1261] T e402d0a40bee47c5a1c18079e0f7ef9e P 3c2587539cec4459b362f77d0448ccd8 [term 1 FOLLOWER]: Refusing update from remote peer e1fbdfb76fed4aae80b18ce233b2c658: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:56:08.440451 7984 raft_consensus.cc:1261] T e402d0a40bee47c5a1c18079e0f7ef9e P c8ee50a8a25644de84f69451956e709d [term 1 FOLLOWER]: Refusing update from remote peer e1fbdfb76fed4aae80b18ce233b2c658: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:56:08.440822 8296 consensus_queue.cc:1035] T e402d0a40bee47c5a1c18079e0f7ef9e P e1fbdfb76fed4aae80b18ce233b2c658 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8ee50a8a25644de84f69451956e709d" member_type: VOTER last_known_addr { host: "127.8.154.2" port: 46809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:56:08.440956 8299 consensus_queue.cc:1035] T e402d0a40bee47c5a1c18079e0f7ef9e P e1fbdfb76fed4aae80b18ce233b2c658 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c2587539cec4459b362f77d0448ccd8" member_type: VOTER last_known_addr { host: "127.8.154.3" port: 45013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:56:08.443326 8306 mvcc.cc:205] Tried to move back new op lower bound from 7316544587529138176 to 7316544587499880448. Current Snapshot: MvccSnapshot[applied={T|T < 7316544587529138176}]
I20260809 08:56:08.443641 8309 mvcc.cc:205] Tried to move back new op lower bound from 7316544587529138176 to 7316544587499880448. Current Snapshot: MvccSnapshot[applied={T|T < 7316544587529138176}]
I20260809 08:56:08.443987 8310 mvcc.cc:205] Tried to move back new op lower bound from 7316544587529138176 to 7316544587499880448. Current Snapshot: MvccSnapshot[applied={T|T < 7316544587529138176}]
I20260809 08:56:08.445986 7541 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:56:08.446128 7541 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:56:08.446883 7541 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f47a2ce6d0814bf49359f12d75c8ac5e: Sending DeleteTablet for 3 replicas of tablet e402d0a40bee47c5a1c18079e0f7ef9e
I20260809 08:56:08.447228 8152 tablet_service.cc:1514] Processing DeleteTablet for tablet e402d0a40bee47c5a1c18079e0f7ef9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:56:08 UTC) from {username='slave'} at 127.0.0.1:44992
I20260809 08:56:08.447243 7743 tablet_service.cc:1514] Processing DeleteTablet for tablet e402d0a40bee47c5a1c18079e0f7ef9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:56:08 UTC) from {username='slave'} at 127.0.0.1:60416
I20260809 08:56:08.447278 7945 tablet_service.cc:1514] Processing DeleteTablet for tablet e402d0a40bee47c5a1c18079e0f7ef9e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:56:08 UTC) from {username='slave'} at 127.0.0.1:41138
I20260809 08:56:08.447468 8322 tablet_replica.cc:330] T e402d0a40bee47c5a1c18079e0f7ef9e P e1fbdfb76fed4aae80b18ce233b2c658: stopping tablet replica
I20260809 08:56:08.447568 8322 raft_consensus.cc:2229] T e402d0a40bee47c5a1c18079e0f7ef9e P e1fbdfb76fed4aae80b18ce233b2c658 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:56:08.447695 8322 raft_consensus.cc:2258] T e402d0a40bee47c5a1c18079e0f7ef9e P e1fbdfb76fed4aae80b18ce233b2c658 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:56:08.447767 8325 tablet_replica.cc:330] T e402d0a40bee47c5a1c18079e0f7ef9e P c8ee50a8a25644de84f69451956e709d: stopping tablet replica
I20260809 08:56:08.447865 8325 raft_consensus.cc:2229] T e402d0a40bee47c5a1c18079e0f7ef9e P c8ee50a8a25644de84f69451956e709d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:56:08.447882 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 7633
I20260809 08:56:08.447968 8325 raft_consensus.cc:2258] T e402d0a40bee47c5a1c18079e0f7ef9e P c8ee50a8a25644de84f69451956e709d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:56:08.448137 8323 tablet_replica.cc:330] T e402d0a40bee47c5a1c18079e0f7ef9e P 3c2587539cec4459b362f77d0448ccd8: stopping tablet replica
I20260809 08:56:08.448216 8323 raft_consensus.cc:2229] T e402d0a40bee47c5a1c18079e0f7ef9e P 3c2587539cec4459b362f77d0448ccd8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:56:08.448297 8323 raft_consensus.cc:2258] T e402d0a40bee47c5a1c18079e0f7ef9e P 3c2587539cec4459b362f77d0448ccd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:56:08.451066 8323 ts_tablet_manager.cc:1905] T e402d0a40bee47c5a1c18079e0f7ef9e P 3c2587539cec4459b362f77d0448ccd8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:56:08.453954 8325 ts_tablet_manager.cc:1905] T e402d0a40bee47c5a1c18079e0f7ef9e P c8ee50a8a25644de84f69451956e709d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:56:08.454540 8323 ts_tablet_manager.cc:1918] T e402d0a40bee47c5a1c18079e0f7ef9e P 3c2587539cec4459b362f77d0448ccd8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:56:08.454631 8323 log.cc:1196] T e402d0a40bee47c5a1c18079e0f7ef9e P 3c2587539cec4459b362f77d0448ccd8: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-2/wal/wals/e402d0a40bee47c5a1c18079e0f7ef9e
I20260809 08:56:08.454919 8323 ts_tablet_manager.cc:1939] T e402d0a40bee47c5a1c18079e0f7ef9e P 3c2587539cec4459b362f77d0448ccd8: Deleting consensus metadata
I20260809 08:56:08.455404 7520 catalog_manager.cc:4841] TS 3c2587539cec4459b362f77d0448ccd8 (127.8.154.3:45013): tablet e402d0a40bee47c5a1c18079e0f7ef9e (table test-table [id=2ddaf8d442ed46f1842c3d66250c1383]) successfully deleted
I20260809 08:56:08.455902 8325 ts_tablet_manager.cc:1918] T e402d0a40bee47c5a1c18079e0f7ef9e P c8ee50a8a25644de84f69451956e709d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:56:08.456005 8325 log.cc:1196] T e402d0a40bee47c5a1c18079e0f7ef9e P c8ee50a8a25644de84f69451956e709d: Deleting WAL directory at /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265519942966-8808-0/minicluster-data/ts-1/wal/wals/e402d0a40bee47c5a1c18079e0f7ef9e
I20260809 08:56:08.456266 8325 ts_tablet_manager.cc:1939] T e402d0a40bee47c5a1c18079e0f7ef9e P c8ee50a8a25644de84f69451956e709d: Deleting consensus metadata
I20260809 08:56:08.456696 7525 catalog_manager.cc:4841] TS c8ee50a8a25644de84f69451956e709d (127.8.154.2:46809): tablet e402d0a40bee47c5a1c18079e0f7ef9e (table test-table [id=2ddaf8d442ed46f1842c3d66250c1383]) successfully deleted
W20260809 08:56:08.466266 7524 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.8.154.1:33455 (error 108)
I20260809 08:56:08.466528 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 7854
I20260809 08:56:08.471926 8280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:08.466677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.1:33455 (local address 127.0.0.1:60436)
0809 08:56:08.471790 (+ 5113us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:08.471873 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.1:33455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5076}
W20260809 08:56:08.473712 7524 catalog_manager.cc:4584] TS e1fbdfb76fed4aae80b18ce233b2c658 (127.8.154.1:33455): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e402d0a40bee47c5a1c18079e0f7ef9e: Network error: Client connection negotiation failed: client connection to 127.8.154.1:33455: connect: Connection refused (error 111)
I20260809 08:56:08.482283 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 8057
I20260809 08:56:08.504244 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 7495
2026-08-09T08:56:08Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (739 ms)
[----------] 6 tests from EncryptionPolicyTest (9279 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:56:08 dist-test-slave-t3q3 krb5kdc[8329](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:56:08 dist-test-slave-t3q3 krb5kdc[8329](info): set up 2 sockets
Aug 09 08:56:08 dist-test-slave-t3q3 krb5kdc[8329](info): commencing operation
krb5kdc: starting...
W20260809 08:56:09.765983 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:56:09 dist-test-slave-t3q3 krb5kdc[8329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:56:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:56:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 08:56:09.888623 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36965
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36965
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:56:09.972872 8349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:09.973114 8349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:09.973138 8349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:09.974309 8349 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:56:09.974359 8349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:09.974373 8349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:09.974387 8349 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:56:09.974444 8349 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:56:09.975955 8349 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36965
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:09.976253 8349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:09.976500 8349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:09.981475 8363 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:09.981485 8361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:09.981472 8365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:09.981871 8349 server_base.cc:1034] running on GCE node
I20260809 08:56:09.982159 8349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:09.982457 8349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:09.983626 8349 hybrid_clock.cc:648] HybridClock initialized: now 1786265769983603 us; error 34 us; skew 500 ppm
Aug 09 08:56:09 dist-test-slave-t3q3 krb5kdc[8329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265769, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:56:09.990283 8349 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 08:56:09.990826 8349 webserver.cc:460] Webserver started at http://127.0.0.1:41183/ using document root <none> and password file <none>
I20260809 08:56:09.991092 8349 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:09.991164 8349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:09.991318 8349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:09.992499 8349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "72b17d1e72e540828ee178daa225a40e"
format_stamp: "Formatted at 2026-08-09 08:56:09 on dist-test-slave-t3q3"
server_key: "1bb5be2720003554d759c483e197030a"
server_key_iv: "5032cbe19aac2aadbdcff2c4f871ff88"
server_key_version: "encryptionkey@0"
I20260809 08:56:09.992897 8349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "72b17d1e72e540828ee178daa225a40e"
format_stamp: "Formatted at 2026-08-09 08:56:09 on dist-test-slave-t3q3"
server_key: "1bb5be2720003554d759c483e197030a"
server_key_iv: "5032cbe19aac2aadbdcff2c4f871ff88"
server_key_version: "encryptionkey@0"
I20260809 08:56:09.994308 8349 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:56:09.995159 8373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:09.995347 8349 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:56:09.995467 8349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "72b17d1e72e540828ee178daa225a40e"
format_stamp: "Formatted at 2026-08-09 08:56:09 on dist-test-slave-t3q3"
server_key: "1bb5be2720003554d759c483e197030a"
server_key_iv: "5032cbe19aac2aadbdcff2c4f871ff88"
server_key_version: "encryptionkey@0"
I20260809 08:56:09.995553 8349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:10.016315 8349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:10.018323 8349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:10.018497 8349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:10.022868 8349 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36965
I20260809 08:56:10.022922 8456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36965 every 8 connection(s)
I20260809 08:56:10.023332 8349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:56:10.023855 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 8349
I20260809 08:56:10.023985 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:56:10.024212 8808 external_mini_cluster.cc:1428] Setting key 319f940d0a2a1f7efd73eea9cbbd2920
I20260809 08:56:10.024623 8460 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:10.027606 8460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72b17d1e72e540828ee178daa225a40e: Bootstrap starting.
I20260809 08:56:10.028359 8460 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72b17d1e72e540828ee178daa225a40e: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:10.028694 8460 log.cc:824] T 00000000000000000000000000000000 P 72b17d1e72e540828ee178daa225a40e: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:10.029568 8460 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72b17d1e72e540828ee178daa225a40e: No bootstrap required, opened a new log
Aug 09 08:56:10 dist-test-slave-t3q3 krb5kdc[8329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265769, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 08:56:10.032173 8460 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72b17d1e72e540828ee178daa225a40e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72b17d1e72e540828ee178daa225a40e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36965 } }
I20260809 08:56:10.032322 8460 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72b17d1e72e540828ee178daa225a40e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:10.032383 8460 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72b17d1e72e540828ee178daa225a40e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72b17d1e72e540828ee178daa225a40e, State: Initialized, Role: FOLLOWER
I20260809 08:56:10.032539 8460 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72b17d1e72e540828ee178daa225a40e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72b17d1e72e540828ee178daa225a40e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36965 } }
I20260809 08:56:10.032637 8460 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72b17d1e72e540828ee178daa225a40e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:56:10.032698 8460 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72b17d1e72e540828ee178daa225a40e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:56:10.032765 8460 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72b17d1e72e540828ee178daa225a40e [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:10.033596 8460 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72b17d1e72e540828ee178daa225a40e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72b17d1e72e540828ee178daa225a40e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36965 } }
I20260809 08:56:10.033726 8460 leader_election.cc:302] T 00000000000000000000000000000000 P 72b17d1e72e540828ee178daa225a40e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72b17d1e72e540828ee178daa225a40e; no voters:
I20260809 08:56:10.033916 8460 leader_election.cc:288] T 00000000000000000000000000000000 P 72b17d1e72e540828ee178daa225a40e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:56:10.034013 8474 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72b17d1e72e540828ee178daa225a40e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:56:10.034217 8460 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72b17d1e72e540828ee178daa225a40e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:56:10.034258 8474 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72b17d1e72e540828ee178daa225a40e [term 1 LEADER]: Becoming Leader. State: Replica: 72b17d1e72e540828ee178daa225a40e, State: Running, Role: LEADER
I20260809 08:56:10.034351 8474 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72b17d1e72e540828ee178daa225a40e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72b17d1e72e540828ee178daa225a40e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36965 } }
I20260809 08:56:10.034670 8474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72b17d1e72e540828ee178daa225a40e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72b17d1e72e540828ee178daa225a40e. Latest consensus state: current_term: 1 leader_uuid: "72b17d1e72e540828ee178daa225a40e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72b17d1e72e540828ee178daa225a40e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36965 } } }
I20260809 08:56:10.034737 8474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72b17d1e72e540828ee178daa225a40e [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:10.034946 8475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72b17d1e72e540828ee178daa225a40e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72b17d1e72e540828ee178daa225a40e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72b17d1e72e540828ee178daa225a40e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36965 } } }
I20260809 08:56:10.035089 8475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72b17d1e72e540828ee178daa225a40e [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:10.035673 8481 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:56:10.036290 8471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:10.024776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34319 (local address 127.0.0.1:36965)
0809 08:56:10.024992 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:10.025000 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:10.025058 (+ 58us) server_negotiation.cc:407] Connection header received
0809 08:56:10.025752 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:10.025760 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:10.025809 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:10.025964 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:10.026645 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:10.027576 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:10.028389 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:10.028944 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:10.029082 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:10.031832 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:10.031836 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:10.031837 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:10.031851 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:10.034135 (+ 2284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:10.034748 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:10.034751 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:10.034752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:10.034883 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:10.035163 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:10.035165 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:10.035165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:10.035527 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:10.035719 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:10.035870 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 08:56:10.036003 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":99,"threads_started":1}
I20260809 08:56:10.036417 8481 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:56:10.037392 8808 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:56:10.045147 8471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:10.038712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37549 (local address 127.0.0.1:36965)
0809 08:56:10.038745 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:10.038747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:10.038854 (+ 107us) server_negotiation.cc:407] Connection header received
0809 08:56:10.038930 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:10.038931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:10.038938 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:10.038985 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:10.039671 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:10.040221 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:10.040970 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:10.041202 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:10.041288 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:10.042493 (+ 1205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:10.042495 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:10.042495 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:10.042502 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:10.043886 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:10.044327 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:10.044328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:10.044328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:10.044393 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:10.044632 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:10.044633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:10.044633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:10.044748 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:10.044860 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:10.044988 (+ 128us) server_negotiation.cc:299] Negotiation successful
0809 08:56:10.045050 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 08:56:10.048450 8481 catalog_manager.cc:1347] Generated new cluster ID: 3fee8fb5b764436a8e297a54f7a54029
I20260809 08:56:10.048509 8481 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:56:10.056160 8481 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:56:10.056947 8481 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:56:10.068343 8481 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72b17d1e72e540828ee178daa225a40e: Generated new TSK 0
I20260809 08:56:10.068578 8481 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:56:10.077638 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 8349
W20260809 08:56:10.091822 8327 connection.cc:535] client connection to 127.0.0.1:36965 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36965): Connection reset by peer (error 104)
2026-08-09T08:56:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1575 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:56:10 dist-test-slave-t3q3 krb5kdc[8511](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:56:10 dist-test-slave-t3q3 krb5kdc[8511](info): set up 2 sockets
Aug 09 08:56:10 dist-test-slave-t3q3 krb5kdc[8511](info): commencing operation
krb5kdc: starting...
W20260809 08:56:11.338248 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.232s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:56:11 dist-test-slave-t3q3 krb5kdc[8511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265771, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:56:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:56:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 08:56:11.466269 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39311
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39311
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:56:11.548733 8543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:11.548938 8543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:11.548959 8543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:11.550025 8543 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:56:11.550060 8543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:11.550076 8543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:11.550087 8543 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:56:11.550099 8543 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:56:11.551512 8543 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39311
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:11.551703 8543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:11.551867 8543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:11.556757 8552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:11.556792 8555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:11.556826 8543 server_base.cc:1034] running on GCE node
W20260809 08:56:11.556808 8550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:11.557205 8543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:11.557487 8543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:11.558629 8543 hybrid_clock.cc:648] HybridClock initialized: now 1786265771558620 us; error 29 us; skew 500 ppm
Aug 09 08:56:11 dist-test-slave-t3q3 krb5kdc[8511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265771, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:56:11.561226 8543 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 08:56:11.561707 8543 webserver.cc:460] Webserver started at http://127.0.0.1:39237/ using document root <none> and password file <none>
I20260809 08:56:11.561954 8543 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:11.562040 8543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:11.562191 8543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:11.563314 8543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "fdf528a4fbd044c5bc695224904d0a7f"
format_stamp: "Formatted at 2026-08-09 08:56:11 on dist-test-slave-t3q3"
server_key: "e1704af667c7e06dee2f121af46f947e"
server_key_iv: "02766429100f16198bb015d9d643f065"
server_key_version: "encryptionkey@0"
I20260809 08:56:11.563707 8543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "fdf528a4fbd044c5bc695224904d0a7f"
format_stamp: "Formatted at 2026-08-09 08:56:11 on dist-test-slave-t3q3"
server_key: "e1704af667c7e06dee2f121af46f947e"
server_key_iv: "02766429100f16198bb015d9d643f065"
server_key_version: "encryptionkey@0"
I20260809 08:56:11.565177 8543 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:56:11.565953 8570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:11.566126 8543 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:56:11.566238 8543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "fdf528a4fbd044c5bc695224904d0a7f"
format_stamp: "Formatted at 2026-08-09 08:56:11 on dist-test-slave-t3q3"
server_key: "e1704af667c7e06dee2f121af46f947e"
server_key_iv: "02766429100f16198bb015d9d643f065"
server_key_version: "encryptionkey@0"
I20260809 08:56:11.566315 8543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:11.583788 8543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:11.585804 8543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:11.585968 8543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:11.590327 8543 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39311
I20260809 08:56:11.590360 8667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39311 every 8 connection(s)
I20260809 08:56:11.590741 8543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:56:11.591066 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 8543
I20260809 08:56:11.591214 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:56:11.591446 8808 external_mini_cluster.cc:1428] Setting key cb5a60dc4dedca47c4053830de45be54
I20260809 08:56:11.592021 8668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:11.595386 8668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdf528a4fbd044c5bc695224904d0a7f: Bootstrap starting.
I20260809 08:56:11.596020 8668 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fdf528a4fbd044c5bc695224904d0a7f: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:11.596410 8668 log.cc:824] T 00000000000000000000000000000000 P fdf528a4fbd044c5bc695224904d0a7f: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:11.597283 8668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdf528a4fbd044c5bc695224904d0a7f: No bootstrap required, opened a new log
I20260809 08:56:11.597703 8673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:11.592024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44475 (local address 127.0.0.1:39311)
0809 08:56:11.592252 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:11.592263 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:11.592295 (+ 32us) server_negotiation.cc:407] Connection header received
0809 08:56:11.592855 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:11.592865 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:11.592930 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:11.593101 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:11.593285 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:11.594682 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:11.595518 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:11.595949 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:11.596093 (+ 144us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:11.596950 (+ 857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:11.596954 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:11.596954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:56:11.597174 (+ 220us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:11.597214 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:56:11.597230 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:11.597330 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:11.597512 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 08:56:11.597536 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":108,"threads_started":1}
I20260809 08:56:11.598831 8808 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:56:11.599650 8668 raft_consensus.cc:348] T 00000000000000000000000000000000 P fdf528a4fbd044c5bc695224904d0a7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdf528a4fbd044c5bc695224904d0a7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39311 } }
I20260809 08:56:11.599864 8668 raft_consensus.cc:374] T 00000000000000000000000000000000 P fdf528a4fbd044c5bc695224904d0a7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:11.599915 8668 raft_consensus.cc:729] T 00000000000000000000000000000000 P fdf528a4fbd044c5bc695224904d0a7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdf528a4fbd044c5bc695224904d0a7f, State: Initialized, Role: FOLLOWER
I20260809 08:56:11.600051 8668 consensus_queue.cc:260] T 00000000000000000000000000000000 P fdf528a4fbd044c5bc695224904d0a7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdf528a4fbd044c5bc695224904d0a7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39311 } }
I20260809 08:56:11.600171 8668 raft_consensus.cc:388] T 00000000000000000000000000000000 P fdf528a4fbd044c5bc695224904d0a7f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:56:11.600215 8668 raft_consensus.cc:482] T 00000000000000000000000000000000 P fdf528a4fbd044c5bc695224904d0a7f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:56:11.600270 8668 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fdf528a4fbd044c5bc695224904d0a7f [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:11.601037 8668 raft_consensus.cc:504] T 00000000000000000000000000000000 P fdf528a4fbd044c5bc695224904d0a7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdf528a4fbd044c5bc695224904d0a7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39311 } }
I20260809 08:56:11.601153 8668 leader_election.cc:302] T 00000000000000000000000000000000 P fdf528a4fbd044c5bc695224904d0a7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fdf528a4fbd044c5bc695224904d0a7f; no voters:
I20260809 08:56:11.601326 8668 leader_election.cc:288] T 00000000000000000000000000000000 P fdf528a4fbd044c5bc695224904d0a7f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:56:11.601409 8684 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fdf528a4fbd044c5bc695224904d0a7f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:56:11.601562 8684 raft_consensus.cc:686] T 00000000000000000000000000000000 P fdf528a4fbd044c5bc695224904d0a7f [term 1 LEADER]: Becoming Leader. State: Replica: fdf528a4fbd044c5bc695224904d0a7f, State: Running, Role: LEADER
I20260809 08:56:11.601641 8668 sys_catalog.cc:564] T 00000000000000000000000000000000 P fdf528a4fbd044c5bc695224904d0a7f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:56:11.601656 8684 consensus_queue.cc:237] T 00000000000000000000000000000000 P fdf528a4fbd044c5bc695224904d0a7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdf528a4fbd044c5bc695224904d0a7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39311 } }
I20260809 08:56:11.602011 8685 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdf528a4fbd044c5bc695224904d0a7f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fdf528a4fbd044c5bc695224904d0a7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdf528a4fbd044c5bc695224904d0a7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39311 } } }
I20260809 08:56:11.602113 8685 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdf528a4fbd044c5bc695224904d0a7f [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:11.602001 8684 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdf528a4fbd044c5bc695224904d0a7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader fdf528a4fbd044c5bc695224904d0a7f. Latest consensus state: current_term: 1 leader_uuid: "fdf528a4fbd044c5bc695224904d0a7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdf528a4fbd044c5bc695224904d0a7f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39311 } } }
I20260809 08:56:11.602293 8684 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdf528a4fbd044c5bc695224904d0a7f [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:11.602399 8690 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:56:11.603111 8690 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:56:11.603312 8708 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 08:56:11.603958 8673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:11.600285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52689 (local address 127.0.0.1:39311)
0809 08:56:11.600350 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:11.600352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:11.600501 (+ 149us) server_negotiation.cc:407] Connection header received
0809 08:56:11.600524 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:11.600525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:11.600535 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:11.600596 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:11.600687 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:11.601372 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:11.602531 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:11.602817 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:11.602867 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:11.603708 (+ 841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:11.603710 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:11.603710 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:56:11.603715 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:11.603730 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:56:11.603734 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:11.603796 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:11.603881 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 08:56:11.603892 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 08:56:11.604849 8690 catalog_manager.cc:1347] Generated new cluster ID: e47795dcc06a4bcd9706f8fa7dd41247
I20260809 08:56:11.604902 8690 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:56:11.614806 8690 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:56:11.615368 8690 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:56:11.619526 8690 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fdf528a4fbd044c5bc695224904d0a7f: Generated new TSK 0
I20260809 08:56:11.619717 8690 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:56:11.625702 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 8543
W20260809 08:56:11.641824 8508 connection.cc:535] client connection to 127.0.0.1:39311 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39311): Connection reset by peer (error 104)
2026-08-09T08:56:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1549 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:56:11 dist-test-slave-t3q3 krb5kdc[8716](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:56:11 dist-test-slave-t3q3 krb5kdc[8716](info): set up 2 sockets
Aug 09 08:56:11 dist-test-slave-t3q3 krb5kdc[8716](info): commencing operation
krb5kdc: starting...
W20260809 08:56:12.887776 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.232s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:56:12 dist-test-slave-t3q3 krb5kdc[8716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:56:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:56:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 08:56:13.010279 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43129
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43129
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:56:13.093187 8744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:13.093425 8744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:13.093467 8744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:13.094590 8744 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:56:13.094663 8744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:13.094691 8744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:13.094725 8744 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:56:13.094756 8744 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:56:13.096225 8744 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43129
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:13.096503 8744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:13.096730 8744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:13.101824 8753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:13.101842 8752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:13.101876 8744 server_base.cc:1034] running on GCE node
W20260809 08:56:13.101840 8756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:13.102308 8744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:13.102643 8744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:13.103818 8744 hybrid_clock.cc:648] HybridClock initialized: now 1786265773103804 us; error 26 us; skew 500 ppm
Aug 09 08:56:13 dist-test-slave-t3q3 krb5kdc[8716](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265773, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:56:13.106878 8744 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 08:56:13.107379 8744 webserver.cc:460] Webserver started at http://127.0.0.1:34149/ using document root <none> and password file <none>
I20260809 08:56:13.107641 8744 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:13.107692 8744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:13.107849 8744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:13.109087 8744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "7244ace45e8346f4bfee69f6907af258"
format_stamp: "Formatted at 2026-08-09 08:56:13 on dist-test-slave-t3q3"
server_key: "4365e3b896c0e92621407fa799bdde56"
server_key_iv: "26f8b8185aa5c59f922b910b2dd896c9"
server_key_version: "encryptionkey@0"
I20260809 08:56:13.109508 8744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "7244ace45e8346f4bfee69f6907af258"
format_stamp: "Formatted at 2026-08-09 08:56:13 on dist-test-slave-t3q3"
server_key: "4365e3b896c0e92621407fa799bdde56"
server_key_iv: "26f8b8185aa5c59f922b910b2dd896c9"
server_key_version: "encryptionkey@0"
I20260809 08:56:13.110975 8744 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 08:56:13.111723 8768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:13.111897 8744 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:13.112020 8744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "7244ace45e8346f4bfee69f6907af258"
format_stamp: "Formatted at 2026-08-09 08:56:13 on dist-test-slave-t3q3"
server_key: "4365e3b896c0e92621407fa799bdde56"
server_key_iv: "26f8b8185aa5c59f922b910b2dd896c9"
server_key_version: "encryptionkey@0"
I20260809 08:56:13.112134 8744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:13.115805 8744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:13.118362 8744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:13.118546 8744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:13.122750 8744 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43129
I20260809 08:56:13.122781 8862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43129 every 8 connection(s)
I20260809 08:56:13.123142 8744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:56:13.124229 8864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:13.125438 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 8744
I20260809 08:56:13.125578 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:56:13.125768 8808 external_mini_cluster.cc:1428] Setting key 694fc992bceac30c0b6a558db397f47c
I20260809 08:56:13.126724 8864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7244ace45e8346f4bfee69f6907af258: Bootstrap starting.
I20260809 08:56:13.127367 8864 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7244ace45e8346f4bfee69f6907af258: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:13.127683 8864 log.cc:824] T 00000000000000000000000000000000 P 7244ace45e8346f4bfee69f6907af258: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:13.128400 8868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:13.126340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59431 (local address 127.0.0.1:43129)
0809 08:56:13.126538 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:13.126546 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:13.126572 (+ 26us) server_negotiation.cc:407] Connection header received
0809 08:56:13.127060 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:13.127067 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:13.127095 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:13.127223 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:13.127922 (+ 699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:13.127927 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:13.127928 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:56:13.127934 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:13.127981 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:56:13.127998 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:13.128125 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:13.128195 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 08:56:13.128239 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":96,"threads_started":1}
I20260809 08:56:13.128631 8864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7244ace45e8346f4bfee69f6907af258: No bootstrap required, opened a new log
I20260809 08:56:13.129539 8808 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:56:13.131213 8864 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7244ace45e8346f4bfee69f6907af258 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7244ace45e8346f4bfee69f6907af258" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43129 } }
I20260809 08:56:13.131400 8864 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7244ace45e8346f4bfee69f6907af258 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:13.131455 8864 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7244ace45e8346f4bfee69f6907af258 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7244ace45e8346f4bfee69f6907af258, State: Initialized, Role: FOLLOWER
I20260809 08:56:13.131574 8864 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7244ace45e8346f4bfee69f6907af258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7244ace45e8346f4bfee69f6907af258" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43129 } }
I20260809 08:56:13.131659 8864 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7244ace45e8346f4bfee69f6907af258 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:56:13.131681 8864 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7244ace45e8346f4bfee69f6907af258 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:56:13.131750 8864 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7244ace45e8346f4bfee69f6907af258 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:13.132392 8868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:13.131221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48197 (local address 127.0.0.1:43129)
0809 08:56:13.131262 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:13.131264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:13.131350 (+ 86us) server_negotiation.cc:407] Connection header received
0809 08:56:13.131423 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:13.131423 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:13.131428 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:13.131486 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:13.132169 (+ 683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:13.132170 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:13.132170 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:56:13.132175 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:13.132190 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:56:13.132195 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:13.132262 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:13.132300 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 08:56:13.132313 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 08:56:13.132447 8864 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7244ace45e8346f4bfee69f6907af258 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7244ace45e8346f4bfee69f6907af258" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43129 } }
I20260809 08:56:13.132546 8864 leader_election.cc:302] T 00000000000000000000000000000000 P 7244ace45e8346f4bfee69f6907af258 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7244ace45e8346f4bfee69f6907af258; no voters:
I20260809 08:56:13.132722 8864 leader_election.cc:288] T 00000000000000000000000000000000 P 7244ace45e8346f4bfee69f6907af258 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:56:13.132781 8879 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7244ace45e8346f4bfee69f6907af258 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:56:13.132949 8879 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7244ace45e8346f4bfee69f6907af258 [term 1 LEADER]: Becoming Leader. State: Replica: 7244ace45e8346f4bfee69f6907af258, State: Running, Role: LEADER
I20260809 08:56:13.132977 8864 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7244ace45e8346f4bfee69f6907af258 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:56:13.133104 8879 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7244ace45e8346f4bfee69f6907af258 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7244ace45e8346f4bfee69f6907af258" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43129 } }
I20260809 08:56:13.133474 8879 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7244ace45e8346f4bfee69f6907af258 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7244ace45e8346f4bfee69f6907af258. Latest consensus state: current_term: 1 leader_uuid: "7244ace45e8346f4bfee69f6907af258" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7244ace45e8346f4bfee69f6907af258" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43129 } } }
I20260809 08:56:13.133560 8879 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7244ace45e8346f4bfee69f6907af258 [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:13.133486 8883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7244ace45e8346f4bfee69f6907af258 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7244ace45e8346f4bfee69f6907af258" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7244ace45e8346f4bfee69f6907af258" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43129 } } }
I20260809 08:56:13.133611 8883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7244ace45e8346f4bfee69f6907af258 [sys.catalog]: This master's current role is: LEADER
W20260809 08:56:13.134790 8909 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7244ace45e8346f4bfee69f6907af258: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:56:13.134855 8909 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:56:13.134903 8893 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:56:13.135056 8893 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:56:13.136557 8893 catalog_manager.cc:1347] Generated new cluster ID: 5863ba29107b42cabf773345e12688db
I20260809 08:56:13.136610 8893 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:56:13.151376 8893 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:56:13.151605 8893 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:56:13.157886 8893 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7244ace45e8346f4bfee69f6907af258: Generated new TSK 0
I20260809 08:56:13.158106 8893 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:56:13.165228 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 8744
2026-08-09T08:56:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1535 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:56:13 dist-test-slave-t3q3 krb5kdc[8921](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:56:13 dist-test-slave-t3q3 krb5kdc[8921](info): set up 2 sockets
Aug 09 08:56:13 dist-test-slave-t3q3 krb5kdc[8921](info): commencing operation
krb5kdc: starting...
W20260809 08:56:14.471154 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.281s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:56:14 dist-test-slave-t3q3 krb5kdc[8921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:56:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:56:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 08:56:14.596277 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33429
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42131
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:56:14.678892 8941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:14.679139 8941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:14.679183 8941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:14.680325 8941 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:56:14.680395 8941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:14.680419 8941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:14.680447 8941 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:56:14.680478 8941 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:56:14.681924 8941 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42131
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:14.682199 8941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:14.682420 8941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:14.687240 8957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:14.687213 8960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:14.687199 8955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:14.687572 8941 server_base.cc:1034] running on GCE node
I20260809 08:56:14.687855 8941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:14.688210 8941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:14.689378 8941 hybrid_clock.cc:648] HybridClock initialized: now 1786265774689357 us; error 32 us; skew 500 ppm
Aug 09 08:56:14 dist-test-slave-t3q3 krb5kdc[8921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265774, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:56:14.691781 8941 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 08:56:14.692241 8941 webserver.cc:460] Webserver started at http://127.0.0.1:45581/ using document root <none> and password file <none>
I20260809 08:56:14.692425 8941 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:14.692464 8941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:14.692551 8941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:14.693640 8941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "f8221ff3f518411fa6e1269f67367213"
format_stamp: "Formatted at 2026-08-09 08:56:14 on dist-test-slave-t3q3"
server_key: "cc17efba8d51a3358273b15f29cced73"
server_key_iv: "4a91c2859115e51fdc07a5bc3b0ac2ec"
server_key_version: "encryptionkey@0"
I20260809 08:56:14.693984 8941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "f8221ff3f518411fa6e1269f67367213"
format_stamp: "Formatted at 2026-08-09 08:56:14 on dist-test-slave-t3q3"
server_key: "cc17efba8d51a3358273b15f29cced73"
server_key_iv: "4a91c2859115e51fdc07a5bc3b0ac2ec"
server_key_version: "encryptionkey@0"
I20260809 08:56:14.695240 8941 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:56:14.695894 8970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:14.696149 8941 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:56:14.696238 8941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "f8221ff3f518411fa6e1269f67367213"
format_stamp: "Formatted at 2026-08-09 08:56:14 on dist-test-slave-t3q3"
server_key: "cc17efba8d51a3358273b15f29cced73"
server_key_iv: "4a91c2859115e51fdc07a5bc3b0ac2ec"
server_key_version: "encryptionkey@0"
I20260809 08:56:14.696292 8941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:14.717320 8941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:14.719295 8941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:14.719408 8941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:14.723663 8941 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33429
I20260809 08:56:14.723721 9049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33429 every 8 connection(s)
I20260809 08:56:14.724117 8941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:56:14.725203 9050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:14.727562 9050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8221ff3f518411fa6e1269f67367213: Bootstrap starting.
I20260809 08:56:14.728253 9050 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8221ff3f518411fa6e1269f67367213: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:14.728648 9050 log.cc:824] T 00000000000000000000000000000000 P f8221ff3f518411fa6e1269f67367213: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:14.729653 9050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8221ff3f518411fa6e1269f67367213: No bootstrap required, opened a new log
I20260809 08:56:14.731691 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 8941
I20260809 08:56:14.731892 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:56:14.732178 8808 external_mini_cluster.cc:1428] Setting key e63dc590a77b891fa8599b7503e6c759
I20260809 08:56:14.732547 9050 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8221ff3f518411fa6e1269f67367213 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8221ff3f518411fa6e1269f67367213" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33429 } }
I20260809 08:56:14.732734 9050 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8221ff3f518411fa6e1269f67367213 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:14.732776 9050 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8221ff3f518411fa6e1269f67367213 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8221ff3f518411fa6e1269f67367213, State: Initialized, Role: FOLLOWER
I20260809 08:56:14.732925 9050 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8221ff3f518411fa6e1269f67367213 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8221ff3f518411fa6e1269f67367213" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33429 } }
I20260809 08:56:14.733057 9050 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8221ff3f518411fa6e1269f67367213 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:56:14.733104 9050 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8221ff3f518411fa6e1269f67367213 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:56:14.733163 9050 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8221ff3f518411fa6e1269f67367213 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:14.734465 9050 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8221ff3f518411fa6e1269f67367213 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8221ff3f518411fa6e1269f67367213" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33429 } }
I20260809 08:56:14.734586 9050 leader_election.cc:302] T 00000000000000000000000000000000 P f8221ff3f518411fa6e1269f67367213 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8221ff3f518411fa6e1269f67367213; no voters:
I20260809 08:56:14.734751 9050 leader_election.cc:288] T 00000000000000000000000000000000 P f8221ff3f518411fa6e1269f67367213 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:56:14.734853 9061 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8221ff3f518411fa6e1269f67367213 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:56:14.735006 9050 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8221ff3f518411fa6e1269f67367213 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:56:14.735047 9061 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8221ff3f518411fa6e1269f67367213 [term 1 LEADER]: Becoming Leader. State: Replica: f8221ff3f518411fa6e1269f67367213, State: Running, Role: LEADER
I20260809 08:56:14.735227 9061 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8221ff3f518411fa6e1269f67367213 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8221ff3f518411fa6e1269f67367213" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33429 } }
I20260809 08:56:14.735646 9061 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8221ff3f518411fa6e1269f67367213 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8221ff3f518411fa6e1269f67367213. Latest consensus state: current_term: 1 leader_uuid: "f8221ff3f518411fa6e1269f67367213" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8221ff3f518411fa6e1269f67367213" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33429 } } }
I20260809 08:56:14.735742 9061 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8221ff3f518411fa6e1269f67367213 [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:14.735652 9062 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8221ff3f518411fa6e1269f67367213 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f8221ff3f518411fa6e1269f67367213" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8221ff3f518411fa6e1269f67367213" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33429 } } }
I20260809 08:56:14.735837 9062 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8221ff3f518411fa6e1269f67367213 [sys.catalog]: This master's current role is: LEADER
W20260809 08:56:14.737200 9082 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f8221ff3f518411fa6e1269f67367213: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:56:14.737262 9082 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:56:14.737311 9075 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:56:14.737484 9075 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:56:14.738989 9075 catalog_manager.cc:1347] Generated new cluster ID: a099366813dd40cfbe3102144c324a6d
I20260809 08:56:14.739042 9075 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 08:56:14 dist-test-slave-t3q3 krb5kdc[8921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265774, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 08:56:14.745226 9058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:14.732888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57855 (local address 127.0.0.1:33429)
0809 08:56:14.733155 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:14.733164 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:14.733198 (+ 34us) server_negotiation.cc:407] Connection header received
0809 08:56:14.733334 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:14.733342 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:14.733399 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:14.733585 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:14.734453 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:14.736023 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:14.737179 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:14.737767 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:14.737892 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:14.740841 (+ 2949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:14.740845 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:14.740846 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:14.740859 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:14.743036 (+ 2177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:14.743459 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:14.743463 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:14.743463 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:14.743594 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:14.743817 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:14.743818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:14.743818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:14.744181 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:14.744343 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:14.744812 (+ 469us) server_negotiation.cc:299] Negotiation successful
0809 08:56:14.744964 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":153,"threads_started":1}
I20260809 08:56:14.746194 8808 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:56:14.749924 9075 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:56:14.750465 9075 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:56:14.754889 9058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:14.747645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37695 (local address 127.0.0.1:33429)
0809 08:56:14.747681 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:14.747683 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:14.747750 (+ 67us) server_negotiation.cc:407] Connection header received
0809 08:56:14.747804 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:14.747805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:14.747815 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:14.747879 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:14.748552 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:14.749266 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:14.750211 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:14.750589 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:14.750663 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:14.752011 (+ 1348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:14.752013 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:14.752013 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:14.752020 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:14.753713 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:14.754096 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:14.754097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:14.754097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:14.754161 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:14.754361 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:14.754362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:14.754363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:14.754493 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:14.754598 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:14.754702 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 08:56:14.754795 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 08:56:14.756412 9075 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8221ff3f518411fa6e1269f67367213: Generated new TSK 0
I20260809 08:56:14.756599 9075 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:56:14.759683 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 8941
2026-08-09T08:56:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1596 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:56:14 dist-test-slave-t3q3 krb5kdc[9098](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:56:14 dist-test-slave-t3q3 krb5kdc[9098](info): set up 2 sockets
Aug 09 08:56:14 dist-test-slave-t3q3 krb5kdc[9098](info): commencing operation
krb5kdc: starting...
W20260809 08:56:16.029811 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.243s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:56:16 dist-test-slave-t3q3 krb5kdc[9098](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265776, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:56:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:56:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 08:56:16.152595 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36017
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36017
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:56:16.234613 9119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:16.234849 9119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:16.234885 9119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:16.236007 9119 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:56:16.236080 9119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:16.236133 9119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:16.236191 9119 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:56:16.236214 9119 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:56:16.237665 9119 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36017
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:16.237936 9119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:16.238176 9119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:16.243165 9129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:16.243325 9119 server_base.cc:1034] running on GCE node
W20260809 08:56:16.243176 9133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:16.243418 9128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:16.243705 9119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:16.244002 9119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:16.245191 9119 hybrid_clock.cc:648] HybridClock initialized: now 1786265776245180 us; error 30 us; skew 500 ppm
Aug 09 08:56:16 dist-test-slave-t3q3 krb5kdc[9098](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265776, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:56:16.250929 9119 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 08:56:16.251394 9119 webserver.cc:460] Webserver started at http://127.0.0.1:39947/ using document root <none> and password file <none>
I20260809 08:56:16.251605 9119 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:16.251646 9119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:16.251737 9119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:16.252779 9119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "5fce912a545340b5bfefaab8ad89fff1"
format_stamp: "Formatted at 2026-08-09 08:56:16 on dist-test-slave-t3q3"
server_key: "82d9c8f3f10504460610265a9e64ffb0"
server_key_iv: "803ba6069a2fb10ece9fd14518315ec3"
server_key_version: "encryptionkey@0"
I20260809 08:56:16.253089 9119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "5fce912a545340b5bfefaab8ad89fff1"
format_stamp: "Formatted at 2026-08-09 08:56:16 on dist-test-slave-t3q3"
server_key: "82d9c8f3f10504460610265a9e64ffb0"
server_key_iv: "803ba6069a2fb10ece9fd14518315ec3"
server_key_version: "encryptionkey@0"
I20260809 08:56:16.254446 9119 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:56:16.255151 9145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:16.255347 9119 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:56:16.255436 9119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "5fce912a545340b5bfefaab8ad89fff1"
format_stamp: "Formatted at 2026-08-09 08:56:16 on dist-test-slave-t3q3"
server_key: "82d9c8f3f10504460610265a9e64ffb0"
server_key_iv: "803ba6069a2fb10ece9fd14518315ec3"
server_key_version: "encryptionkey@0"
I20260809 08:56:16.255489 9119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:16.272804 9119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:16.275033 9119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:16.275169 9119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:16.279575 9119 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36017
I20260809 08:56:16.279627 9234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36017 every 8 connection(s)
I20260809 08:56:16.279985 9119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:56:16.281188 9237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:16.283488 9237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fce912a545340b5bfefaab8ad89fff1: Bootstrap starting.
I20260809 08:56:16.284179 9237 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5fce912a545340b5bfefaab8ad89fff1: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:16.284469 9237 log.cc:824] T 00000000000000000000000000000000 P 5fce912a545340b5bfefaab8ad89fff1: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:16.285324 9237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fce912a545340b5bfefaab8ad89fff1: No bootstrap required, opened a new log
I20260809 08:56:16.286953 9237 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5fce912a545340b5bfefaab8ad89fff1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fce912a545340b5bfefaab8ad89fff1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36017 } }
I20260809 08:56:16.287127 9237 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5fce912a545340b5bfefaab8ad89fff1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:16.287189 9237 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5fce912a545340b5bfefaab8ad89fff1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fce912a545340b5bfefaab8ad89fff1, State: Initialized, Role: FOLLOWER
I20260809 08:56:16.287374 9237 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5fce912a545340b5bfefaab8ad89fff1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fce912a545340b5bfefaab8ad89fff1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36017 } }
I20260809 08:56:16.287492 9237 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5fce912a545340b5bfefaab8ad89fff1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:56:16.287539 9237 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5fce912a545340b5bfefaab8ad89fff1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:56:16.287590 9237 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5fce912a545340b5bfefaab8ad89fff1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:16.287665 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 9119
I20260809 08:56:16.287809 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:56:16.288110 8808 external_mini_cluster.cc:1428] Setting key a8f3e2d9db2f2e6c2c3a0c70b44ed59a
I20260809 08:56:16.288370 9237 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5fce912a545340b5bfefaab8ad89fff1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fce912a545340b5bfefaab8ad89fff1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36017 } }
I20260809 08:56:16.288496 9237 leader_election.cc:302] T 00000000000000000000000000000000 P 5fce912a545340b5bfefaab8ad89fff1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5fce912a545340b5bfefaab8ad89fff1; no voters:
I20260809 08:56:16.288669 9237 leader_election.cc:288] T 00000000000000000000000000000000 P 5fce912a545340b5bfefaab8ad89fff1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:56:16.288935 9237 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5fce912a545340b5bfefaab8ad89fff1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:56:16.288944 9242 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5fce912a545340b5bfefaab8ad89fff1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:56:16.289181 9242 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5fce912a545340b5bfefaab8ad89fff1 [term 1 LEADER]: Becoming Leader. State: Replica: 5fce912a545340b5bfefaab8ad89fff1, State: Running, Role: LEADER
I20260809 08:56:16.289279 9242 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5fce912a545340b5bfefaab8ad89fff1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fce912a545340b5bfefaab8ad89fff1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36017 } }
I20260809 08:56:16.289824 9244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fce912a545340b5bfefaab8ad89fff1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5fce912a545340b5bfefaab8ad89fff1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fce912a545340b5bfefaab8ad89fff1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36017 } } }
I20260809 08:56:16.289925 9244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fce912a545340b5bfefaab8ad89fff1 [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:16.290158 9261 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:56:16.290128 9247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fce912a545340b5bfefaab8ad89fff1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5fce912a545340b5bfefaab8ad89fff1. Latest consensus state: current_term: 1 leader_uuid: "5fce912a545340b5bfefaab8ad89fff1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fce912a545340b5bfefaab8ad89fff1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36017 } } }
I20260809 08:56:16.290269 9247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fce912a545340b5bfefaab8ad89fff1 [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:16.291153 9261 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:56:16.292936 9261 catalog_manager.cc:1347] Generated new cluster ID: 1db1567ad4f8446bbc55d078161a79e3
I20260809 08:56:16.293025 9261 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:56:16.294987 9243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:16.288634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48691 (local address 127.0.0.1:36017)
0809 08:56:16.288835 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:16.288845 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:16.288882 (+ 37us) server_negotiation.cc:407] Connection header received
0809 08:56:16.289084 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:16.289139 (+ 55us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:16.289212 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:16.289770 (+ 558us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:16.289889 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:16.291401 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:16.292515 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:16.293151 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:16.293623 (+ 472us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:16.294061 (+ 438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:16.294065 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:16.294065 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:56:16.294271 (+ 206us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:16.294311 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:56:16.294326 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:16.294647 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:16.294680 (+ 33us) server_negotiation.cc:299] Negotiation successful
0809 08:56:16.294749 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":99,"threads_started":1}
I20260809 08:56:16.295830 8808 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:56:16.300959 9243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:16.297412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35417 (local address 127.0.0.1:36017)
0809 08:56:16.297444 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:16.297446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:16.297554 (+ 108us) server_negotiation.cc:407] Connection header received
0809 08:56:16.297609 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:16.297610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:16.297621 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:16.297674 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:16.297821 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:16.298525 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:16.299601 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:16.299928 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:16.300003 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:16.300718 (+ 715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:16.300719 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:16.300719 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:56:16.300724 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:16.300741 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:56:16.300746 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:16.300807 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:16.300864 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 08:56:16.300887 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 08:56:16.326484 9261 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:56:16.327261 9261 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:56:16.338174 9261 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5fce912a545340b5bfefaab8ad89fff1: Generated new TSK 0
I20260809 08:56:16.338382 9261 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:56:16.346881 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 9119
2026-08-09T08:56:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1583 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:56:16 dist-test-slave-t3q3 krb5kdc[9284](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:56:16 dist-test-slave-t3q3 krb5kdc[9284](info): set up 2 sockets
Aug 09 08:56:16 dist-test-slave-t3q3 krb5kdc[9284](info): commencing operation
krb5kdc: starting...
W20260809 08:56:17.594688 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.224s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:56:17 dist-test-slave-t3q3 krb5kdc[9284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265777, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:56:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:56:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 08:56:17.735404 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46015
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46015
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:56:17.820528 9305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:17.820749 9305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:17.820772 9305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:17.821939 9305 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:56:17.821990 9305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:17.822005 9305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:17.822018 9305 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:56:17.822031 9305 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:56:17.823530 9305 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46015
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:17.823794 9305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:17.824025 9305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:17.829041 9316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:17.829017 9318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:17.829007 9314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:17.829337 9305 server_base.cc:1034] running on GCE node
I20260809 08:56:17.829609 9305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:17.829907 9305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:17.831058 9305 hybrid_clock.cc:648] HybridClock initialized: now 1786265777831041 us; error 27 us; skew 500 ppm
Aug 09 08:56:17 dist-test-slave-t3q3 krb5kdc[9284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265777, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:56:17.833567 9305 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 08:56:17.834048 9305 webserver.cc:460] Webserver started at http://127.0.0.1:43691/ using document root <none> and password file <none>
I20260809 08:56:17.834250 9305 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:17.834292 9305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:17.834381 9305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:17.835363 9305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "005f16dc3c4e46c592bbdb022c1fdb8d"
format_stamp: "Formatted at 2026-08-09 08:56:17 on dist-test-slave-t3q3"
server_key: "e05ad3b6e35f07478a323181ad65cedd"
server_key_iv: "39afa11f39868ad35b506c6e4237901a"
server_key_version: "encryptionkey@0"
I20260809 08:56:17.835673 9305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "005f16dc3c4e46c592bbdb022c1fdb8d"
format_stamp: "Formatted at 2026-08-09 08:56:17 on dist-test-slave-t3q3"
server_key: "e05ad3b6e35f07478a323181ad65cedd"
server_key_iv: "39afa11f39868ad35b506c6e4237901a"
server_key_version: "encryptionkey@0"
I20260809 08:56:17.837061 9305 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:56:17.837759 9334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:17.837950 9305 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:56:17.838038 9305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "005f16dc3c4e46c592bbdb022c1fdb8d"
format_stamp: "Formatted at 2026-08-09 08:56:17 on dist-test-slave-t3q3"
server_key: "e05ad3b6e35f07478a323181ad65cedd"
server_key_iv: "39afa11f39868ad35b506c6e4237901a"
server_key_version: "encryptionkey@0"
I20260809 08:56:17.838093 9305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:17.841485 9305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:17.843461 9305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:17.843609 9305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:17.847659 9305 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46015
I20260809 08:56:17.847718 9420 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46015 every 8 connection(s)
I20260809 08:56:17.848048 9305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:56:17.849141 9421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:17.850976 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 9305
I20260809 08:56:17.851105 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:56:17.851290 8808 external_mini_cluster.cc:1428] Setting key ca70f99cc9752d6da0181bab874fe4f7
I20260809 08:56:17.851377 9421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 005f16dc3c4e46c592bbdb022c1fdb8d: Bootstrap starting.
I20260809 08:56:17.852171 9421 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 005f16dc3c4e46c592bbdb022c1fdb8d: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:17.852504 9421 log.cc:824] T 00000000000000000000000000000000 P 005f16dc3c4e46c592bbdb022c1fdb8d: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:17.853329 9421 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 005f16dc3c4e46c592bbdb022c1fdb8d: No bootstrap required, opened a new log
I20260809 08:56:17.854104 9429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:17.851819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40907 (local address 127.0.0.1:46015)
0809 08:56:17.852031 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:17.852040 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:17.852103 (+ 63us) server_negotiation.cc:407] Connection header received
0809 08:56:17.852766 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:17.852776 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:17.852815 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:17.852931 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:17.853629 (+ 698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:17.853632 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:17.853632 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:56:17.853636 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:17.853694 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:56:17.853705 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:17.853781 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:17.853853 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 08:56:17.853904 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":83,"threads_started":1}
I20260809 08:56:17.855118 8808 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:56:17.855705 9421 raft_consensus.cc:348] T 00000000000000000000000000000000 P 005f16dc3c4e46c592bbdb022c1fdb8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "005f16dc3c4e46c592bbdb022c1fdb8d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46015 } }
I20260809 08:56:17.855861 9421 raft_consensus.cc:374] T 00000000000000000000000000000000 P 005f16dc3c4e46c592bbdb022c1fdb8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:17.855908 9421 raft_consensus.cc:729] T 00000000000000000000000000000000 P 005f16dc3c4e46c592bbdb022c1fdb8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 005f16dc3c4e46c592bbdb022c1fdb8d, State: Initialized, Role: FOLLOWER
I20260809 08:56:17.856026 9421 consensus_queue.cc:260] T 00000000000000000000000000000000 P 005f16dc3c4e46c592bbdb022c1fdb8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "005f16dc3c4e46c592bbdb022c1fdb8d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46015 } }
I20260809 08:56:17.856221 9421 raft_consensus.cc:388] T 00000000000000000000000000000000 P 005f16dc3c4e46c592bbdb022c1fdb8d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:56:17.856274 9421 raft_consensus.cc:482] T 00000000000000000000000000000000 P 005f16dc3c4e46c592bbdb022c1fdb8d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:56:17.856330 9421 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 005f16dc3c4e46c592bbdb022c1fdb8d [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:17.857131 9421 raft_consensus.cc:504] T 00000000000000000000000000000000 P 005f16dc3c4e46c592bbdb022c1fdb8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "005f16dc3c4e46c592bbdb022c1fdb8d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46015 } }
I20260809 08:56:17.857266 9421 leader_election.cc:302] T 00000000000000000000000000000000 P 005f16dc3c4e46c592bbdb022c1fdb8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 005f16dc3c4e46c592bbdb022c1fdb8d; no voters:
I20260809 08:56:17.857448 9421 leader_election.cc:288] T 00000000000000000000000000000000 P 005f16dc3c4e46c592bbdb022c1fdb8d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:56:17.857487 9442 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 005f16dc3c4e46c592bbdb022c1fdb8d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:56:17.857667 9442 raft_consensus.cc:686] T 00000000000000000000000000000000 P 005f16dc3c4e46c592bbdb022c1fdb8d [term 1 LEADER]: Becoming Leader. State: Replica: 005f16dc3c4e46c592bbdb022c1fdb8d, State: Running, Role: LEADER
I20260809 08:56:17.857717 9421 sys_catalog.cc:564] T 00000000000000000000000000000000 P 005f16dc3c4e46c592bbdb022c1fdb8d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:56:17.857775 9429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:17.856617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44615 (local address 127.0.0.1:46015)
0809 08:56:17.856654 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:17.856656 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:17.856696 (+ 40us) server_negotiation.cc:407] Connection header received
0809 08:56:17.856768 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:17.856769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:17.856772 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:17.856812 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:17.857405 (+ 593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:17.857406 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:17.857407 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:56:17.857411 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:17.857426 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:56:17.857431 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:17.857479 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:17.857632 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 08:56:17.857646 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 08:56:17.857785 9442 consensus_queue.cc:237] T 00000000000000000000000000000000 P 005f16dc3c4e46c592bbdb022c1fdb8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "005f16dc3c4e46c592bbdb022c1fdb8d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46015 } }
I20260809 08:56:17.858470 9442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 005f16dc3c4e46c592bbdb022c1fdb8d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "005f16dc3c4e46c592bbdb022c1fdb8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "005f16dc3c4e46c592bbdb022c1fdb8d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46015 } } }
I20260809 08:56:17.858553 9442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 005f16dc3c4e46c592bbdb022c1fdb8d [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:17.858692 9442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 005f16dc3c4e46c592bbdb022c1fdb8d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 005f16dc3c4e46c592bbdb022c1fdb8d. Latest consensus state: current_term: 1 leader_uuid: "005f16dc3c4e46c592bbdb022c1fdb8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "005f16dc3c4e46c592bbdb022c1fdb8d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46015 } } }
I20260809 08:56:17.858763 9442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 005f16dc3c4e46c592bbdb022c1fdb8d [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:17.858865 9457 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:56:17.859381 9457 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:56:17.860929 9457 catalog_manager.cc:1347] Generated new cluster ID: 0b2aa369a7494c2bb7d754fedce67b72
I20260809 08:56:17.861001 9457 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:56:17.866110 9457 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:56:17.866263 9457 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:56:17.869395 9457 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 005f16dc3c4e46c592bbdb022c1fdb8d: Generated new TSK 0
I20260809 08:56:17.869551 9457 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:56:17.875415 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 9305
2026-08-09T08:56:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1526 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:56:17 dist-test-slave-t3q3 krb5kdc[9470](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:56:17 dist-test-slave-t3q3 krb5kdc[9470](info): set up 2 sockets
Aug 09 08:56:17 dist-test-slave-t3q3 krb5kdc[9470](info): commencing operation
krb5kdc: starting...
W20260809 08:56:19.118351 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.222s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:56:19 dist-test-slave-t3q3 krb5kdc[9470](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265779, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:56:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:56:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 08:56:19.242723 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36563
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36563
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:56:19.323566 9492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:19.323751 9492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:19.323773 9492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:19.324827 9492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:56:19.324867 9492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:19.324882 9492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:19.324893 9492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:56:19.324905 9492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:56:19.326277 9492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36563
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:19.326452 9492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:19.326612 9492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:19.331310 9502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:19.331359 9499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:19.331339 9498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:19.331615 9492 server_base.cc:1034] running on GCE node
I20260809 08:56:19.331879 9492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:19.332173 9492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:19.333314 9492 hybrid_clock.cc:648] HybridClock initialized: now 1786265779333304 us; error 26 us; skew 500 ppm
Aug 09 08:56:19 dist-test-slave-t3q3 krb5kdc[9470](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265779, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:56:19.335698 9492 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 08:56:19.336227 9492 webserver.cc:460] Webserver started at http://127.0.0.1:45299/ using document root <none> and password file <none>
I20260809 08:56:19.336473 9492 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:19.336517 9492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:19.336666 9492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:19.337649 9492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "73d53c3d90484f809f1312d7df748cef"
format_stamp: "Formatted at 2026-08-09 08:56:19 on dist-test-slave-t3q3"
server_key: "f3457e655a2e8acdbf60cc55dff2cd22"
server_key_iv: "6c51221bca94f01441abc61b0eb4f3ef"
server_key_version: "encryptionkey@0"
I20260809 08:56:19.337993 9492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "73d53c3d90484f809f1312d7df748cef"
format_stamp: "Formatted at 2026-08-09 08:56:19 on dist-test-slave-t3q3"
server_key: "f3457e655a2e8acdbf60cc55dff2cd22"
server_key_iv: "6c51221bca94f01441abc61b0eb4f3ef"
server_key_version: "encryptionkey@0"
I20260809 08:56:19.339282 9492 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:56:19.339987 9512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:19.340153 9492 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:56:19.340250 9492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "73d53c3d90484f809f1312d7df748cef"
format_stamp: "Formatted at 2026-08-09 08:56:19 on dist-test-slave-t3q3"
server_key: "f3457e655a2e8acdbf60cc55dff2cd22"
server_key_iv: "6c51221bca94f01441abc61b0eb4f3ef"
server_key_version: "encryptionkey@0"
I20260809 08:56:19.340302 9492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:19.363677 9492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:19.366010 9492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:19.366180 9492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:19.370606 9492 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36563
I20260809 08:56:19.370663 9619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36563 every 8 connection(s)
I20260809 08:56:19.371034 9492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:56:19.372336 9620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:19.375909 9620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73d53c3d90484f809f1312d7df748cef: Bootstrap starting.
I20260809 08:56:19.376700 9620 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73d53c3d90484f809f1312d7df748cef: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:19.377105 9620 log.cc:824] T 00000000000000000000000000000000 P 73d53c3d90484f809f1312d7df748cef: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:19.377442 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 9492
I20260809 08:56:19.377561 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:56:19.377750 8808 external_mini_cluster.cc:1428] Setting key d96f544f7004a0e7954ae67ff5d8e708
I20260809 08:56:19.377976 9620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73d53c3d90484f809f1312d7df748cef: No bootstrap required, opened a new log
I20260809 08:56:19.381350 9620 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73d53c3d90484f809f1312d7df748cef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73d53c3d90484f809f1312d7df748cef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36563 } }
I20260809 08:56:19.381606 9620 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73d53c3d90484f809f1312d7df748cef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:19.381703 9620 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73d53c3d90484f809f1312d7df748cef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73d53c3d90484f809f1312d7df748cef, State: Initialized, Role: FOLLOWER
I20260809 08:56:19.381898 9620 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73d53c3d90484f809f1312d7df748cef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73d53c3d90484f809f1312d7df748cef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36563 } }
I20260809 08:56:19.382081 9620 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73d53c3d90484f809f1312d7df748cef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:56:19.382153 9620 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73d53c3d90484f809f1312d7df748cef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:56:19.382237 9620 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73d53c3d90484f809f1312d7df748cef [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:19.382977 9620 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73d53c3d90484f809f1312d7df748cef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73d53c3d90484f809f1312d7df748cef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36563 } }
I20260809 08:56:19.383102 9620 leader_election.cc:302] T 00000000000000000000000000000000 P 73d53c3d90484f809f1312d7df748cef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73d53c3d90484f809f1312d7df748cef; no voters:
I20260809 08:56:19.383297 9620 leader_election.cc:288] T 00000000000000000000000000000000 P 73d53c3d90484f809f1312d7df748cef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:56:19.383366 9626 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73d53c3d90484f809f1312d7df748cef [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:56:19.383909 9626 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73d53c3d90484f809f1312d7df748cef [term 1 LEADER]: Becoming Leader. State: Replica: 73d53c3d90484f809f1312d7df748cef, State: Running, Role: LEADER
I20260809 08:56:19.384004 9626 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73d53c3d90484f809f1312d7df748cef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73d53c3d90484f809f1312d7df748cef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36563 } }
I20260809 08:56:19.384191 9620 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73d53c3d90484f809f1312d7df748cef [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:56:19.384338 9626 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73d53c3d90484f809f1312d7df748cef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73d53c3d90484f809f1312d7df748cef. Latest consensus state: current_term: 1 leader_uuid: "73d53c3d90484f809f1312d7df748cef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73d53c3d90484f809f1312d7df748cef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36563 } } }
I20260809 08:56:19.384344 9627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73d53c3d90484f809f1312d7df748cef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "73d53c3d90484f809f1312d7df748cef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73d53c3d90484f809f1312d7df748cef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36563 } } }
I20260809 08:56:19.384431 9626 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73d53c3d90484f809f1312d7df748cef [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:19.384436 9627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73d53c3d90484f809f1312d7df748cef [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:19.384639 9635 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:56:19.385113 9635 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:56:19.388302 9635 catalog_manager.cc:1347] Generated new cluster ID: ee218b7816954d3f97c5a6e7d5f0188a
I20260809 08:56:19.388355 9635 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 08:56:19 dist-test-slave-t3q3 krb5kdc[9470](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265779, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 08:56:19.393728 9625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:19.380519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:39509 (local address 127.0.0.1:36563)
0809 08:56:19.380748 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:19.380760 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:19.380801 (+ 41us) server_negotiation.cc:407] Connection header received
0809 08:56:19.381307 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:19.381315 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:19.381381 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:19.381568 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:19.382409 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:19.383914 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:19.385654 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:19.386273 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:19.386772 (+ 499us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:19.389027 (+ 2255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:19.389032 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:19.389032 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:19.389045 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:19.391369 (+ 2324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:19.391874 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:19.391877 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:19.391878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:19.392043 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:19.392357 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:19.392359 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:19.392359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:19.392696 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:19.392869 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:19.393366 (+ 497us) server_negotiation.cc:299] Negotiation successful
0809 08:56:19.393485 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":117,"threads_started":1}
I20260809 08:56:19.394307 9635 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:56:19.394990 8808 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:56:19.395040 9635 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:56:19.406824 9635 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 73d53c3d90484f809f1312d7df748cef: Generated new TSK 0
I20260809 08:56:19.407274 9635 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:56:19.407968 9625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:19.396527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:42413 (local address 127.0.0.1:36563)
0809 08:56:19.396563 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:19.396564 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:19.400265 (+ 3701us) server_negotiation.cc:407] Connection header received
0809 08:56:19.400363 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:19.400364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:19.400373 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:19.400402 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:19.401087 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:19.401887 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:19.402974 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:19.403326 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:19.403366 (+ 40us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:19.404859 (+ 1493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:19.404861 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:19.404861 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:19.404868 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:19.406406 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:19.406830 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:19.406831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:19.406832 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:19.406923 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:19.407288 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:19.407289 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:19.407289 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:19.407428 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:19.407506 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:19.407770 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 08:56:19.407853 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:56:19.410445 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 9492
W20260809 08:56:19.430876 9468 connection.cc:535] client connection to 127.0.0.1:36563 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36563): Connection reset by peer (error 104)
2026-08-09T08:56:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1548 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:56:19 dist-test-slave-t3q3 krb5kdc[9669](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:56:19 dist-test-slave-t3q3 krb5kdc[9669](info): set up 2 sockets
Aug 09 08:56:19 dist-test-slave-t3q3 krb5kdc[9669](info): commencing operation
krb5kdc: starting...
W20260809 08:56:20.672361 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.227s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:56:20 dist-test-slave-t3q3 krb5kdc[9669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:56:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:56:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 08:56:20.796838 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34267
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34267
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:56:20.879870 9688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:20.880069 9688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:20.880141 9688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:20.881335 9688 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:56:20.881376 9688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:20.881389 9688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:20.881402 9688 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:56:20.881414 9688 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:56:20.882838 9688 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:20.883025 9688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:20.883191 9688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:20.888118 9701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:20.888039 9703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:20.888186 9699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:20.888322 9688 server_base.cc:1034] running on GCE node
I20260809 08:56:20.888625 9688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:20.888931 9688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:20.890111 9688 hybrid_clock.cc:648] HybridClock initialized: now 1786265780890097 us; error 37 us; skew 500 ppm
Aug 09 08:56:20 dist-test-slave-t3q3 krb5kdc[9669](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265780, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:56:20.892840 9688 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 08:56:20.893354 9688 webserver.cc:460] Webserver started at http://127.0.0.1:41987/ using document root <none> and password file <none>
I20260809 08:56:20.893615 9688 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:20.893661 9688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:20.893751 9688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:20.894726 9688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "5be9c58f93e6422480b202a8b0a166b2"
format_stamp: "Formatted at 2026-08-09 08:56:20 on dist-test-slave-t3q3"
server_key: "a0a42b0c25d24772bde643ec24e171bc"
server_key_iv: "d03d2bd85964860b777dcd10164aed29"
server_key_version: "encryptionkey@0"
I20260809 08:56:20.895027 9688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "5be9c58f93e6422480b202a8b0a166b2"
format_stamp: "Formatted at 2026-08-09 08:56:20 on dist-test-slave-t3q3"
server_key: "a0a42b0c25d24772bde643ec24e171bc"
server_key_iv: "d03d2bd85964860b777dcd10164aed29"
server_key_version: "encryptionkey@0"
I20260809 08:56:20.896308 9688 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 08:56:20.896970 9724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:20.897185 9688 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:20.897272 9688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "5be9c58f93e6422480b202a8b0a166b2"
format_stamp: "Formatted at 2026-08-09 08:56:20 on dist-test-slave-t3q3"
server_key: "a0a42b0c25d24772bde643ec24e171bc"
server_key_iv: "d03d2bd85964860b777dcd10164aed29"
server_key_version: "encryptionkey@0"
I20260809 08:56:20.897325 9688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:20.918593 9688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:20.920660 9688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:20.920782 9688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:20.924841 9688 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34267
I20260809 08:56:20.924916 9805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34267 every 8 connection(s)
I20260809 08:56:20.925275 9688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:56:20.926321 9808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:20.928702 9808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5be9c58f93e6422480b202a8b0a166b2: Bootstrap starting.
I20260809 08:56:20.929296 9808 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5be9c58f93e6422480b202a8b0a166b2: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:20.929586 9808 log.cc:824] T 00000000000000000000000000000000 P 5be9c58f93e6422480b202a8b0a166b2: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:20.930312 9808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5be9c58f93e6422480b202a8b0a166b2: No bootstrap required, opened a new log
I20260809 08:56:20.931983 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 9688
I20260809 08:56:20.931926 9808 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5be9c58f93e6422480b202a8b0a166b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5be9c58f93e6422480b202a8b0a166b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34267 } }
I20260809 08:56:20.932106 9808 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5be9c58f93e6422480b202a8b0a166b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:20.932181 9808 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5be9c58f93e6422480b202a8b0a166b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5be9c58f93e6422480b202a8b0a166b2, State: Initialized, Role: FOLLOWER
I20260809 08:56:20.932197 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:56:20.932420 8808 external_mini_cluster.cc:1428] Setting key 8a8e01260ff86d5897cc69c60ecb5b96
I20260809 08:56:20.932371 9808 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5be9c58f93e6422480b202a8b0a166b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5be9c58f93e6422480b202a8b0a166b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34267 } }
I20260809 08:56:20.932472 9808 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5be9c58f93e6422480b202a8b0a166b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:56:20.932526 9808 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5be9c58f93e6422480b202a8b0a166b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:56:20.932583 9808 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5be9c58f93e6422480b202a8b0a166b2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:20.933319 9808 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5be9c58f93e6422480b202a8b0a166b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5be9c58f93e6422480b202a8b0a166b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34267 } }
I20260809 08:56:20.933446 9808 leader_election.cc:302] T 00000000000000000000000000000000 P 5be9c58f93e6422480b202a8b0a166b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5be9c58f93e6422480b202a8b0a166b2; no voters:
I20260809 08:56:20.933646 9808 leader_election.cc:288] T 00000000000000000000000000000000 P 5be9c58f93e6422480b202a8b0a166b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:56:20.933667 9820 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5be9c58f93e6422480b202a8b0a166b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:56:20.933784 9820 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5be9c58f93e6422480b202a8b0a166b2 [term 1 LEADER]: Becoming Leader. State: Replica: 5be9c58f93e6422480b202a8b0a166b2, State: Running, Role: LEADER
I20260809 08:56:20.933923 9808 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5be9c58f93e6422480b202a8b0a166b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:56:20.933933 9820 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5be9c58f93e6422480b202a8b0a166b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5be9c58f93e6422480b202a8b0a166b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34267 } }
I20260809 08:56:20.934434 9823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5be9c58f93e6422480b202a8b0a166b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5be9c58f93e6422480b202a8b0a166b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5be9c58f93e6422480b202a8b0a166b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34267 } } }
I20260809 08:56:20.934450 9825 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5be9c58f93e6422480b202a8b0a166b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5be9c58f93e6422480b202a8b0a166b2. Latest consensus state: current_term: 1 leader_uuid: "5be9c58f93e6422480b202a8b0a166b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5be9c58f93e6422480b202a8b0a166b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34267 } } }
I20260809 08:56:20.934599 9825 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5be9c58f93e6422480b202a8b0a166b2 [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:20.934551 9823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5be9c58f93e6422480b202a8b0a166b2 [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:20.934862 9836 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:56:20.935415 9836 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:56:20.937171 9836 catalog_manager.cc:1347] Generated new cluster ID: 2464ecdbcbbc409a8623dd79c08ca9ed
I20260809 08:56:20.937227 9836 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:56:20.938879 9818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:20.932955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:47081 (local address 127.0.0.1:34267)
0809 08:56:20.933457 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:20.933465 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:20.933492 (+ 27us) server_negotiation.cc:407] Connection header received
0809 08:56:20.933641 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:20.933646 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:20.933712 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:20.934110 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:20.934211 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:20.935694 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:20.936690 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:20.937293 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:20.937417 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:20.938180 (+ 763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:20.938185 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:20.938185 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:56:20.938399 (+ 214us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:20.938438 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:56:20.938453 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:20.938559 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:20.938643 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 08:56:20.938672 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":409,"threads_started":1}
I20260809 08:56:20.939831 8808 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:56:20.945292 9818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:20.941292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:59677 (local address 127.0.0.1:34267)
0809 08:56:20.941335 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:20.941337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:20.941486 (+ 149us) server_negotiation.cc:407] Connection header received
0809 08:56:20.941597 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:20.941598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:20.941618 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:20.941649 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:20.941846 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:20.942612 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:20.943735 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:20.944162 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:20.944210 (+ 48us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:20.945002 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:20.945003 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:20.945003 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:56:20.945008 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:20.945022 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:56:20.945026 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:20.945060 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:20.945200 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 08:56:20.945216 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 08:56:20.947564 9836 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:56:20.948275 9836 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:56:20.956595 9836 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5be9c58f93e6422480b202a8b0a166b2: Generated new TSK 0
I20260809 08:56:20.956774 9836 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:56:20.964537 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 9688
W20260809 08:56:20.981153 9667 connection.cc:535] client connection to 127.0.0.1:34267 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34267): Connection reset by peer (error 104)
2026-08-09T08:56:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1550 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:56:21 dist-test-slave-t3q3 krb5kdc[9865](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:56:21 dist-test-slave-t3q3 krb5kdc[9865](info): set up 2 sockets
Aug 09 08:56:21 dist-test-slave-t3q3 krb5kdc[9865](info): commencing operation
krb5kdc: starting...
W20260809 08:56:22.237526 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.243s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:56:22 dist-test-slave-t3q3 krb5kdc[9865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:56:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:56:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 08:56:22.363245 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46703
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46703
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:56:22.448109 9889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:22.448311 9889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:22.448335 9889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:22.449446 9889 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:56:22.449483 9889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:22.449497 9889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:22.449510 9889 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:56:22.449522 9889 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:56:22.450927 9889 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46703
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:22.451120 9889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:22.451292 9889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:22.456063 9908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:22.456091 9902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:22.456107 9903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:22.456286 9889 server_base.cc:1034] running on GCE node
I20260809 08:56:22.456664 9889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:22.456951 9889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:22.458096 9889 hybrid_clock.cc:648] HybridClock initialized: now 1786265782458082 us; error 29 us; skew 500 ppm
Aug 09 08:56:22 dist-test-slave-t3q3 krb5kdc[9865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265782, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:56:22.463780 9889 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 08:56:22.464284 9889 webserver.cc:460] Webserver started at http://127.0.0.1:35985/ using document root <none> and password file <none>
I20260809 08:56:22.464545 9889 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:22.464622 9889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:22.464794 9889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:22.465853 9889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "72a3879fd81c4c98bc2d65db2570e946"
format_stamp: "Formatted at 2026-08-09 08:56:22 on dist-test-slave-t3q3"
server_key: "e5db2f27cc4d6b29ceac72e3f6e4e938"
server_key_iv: "c893905a483e91c4daf78c36f85c3ef0"
server_key_version: "encryptionkey@0"
I20260809 08:56:22.466226 9889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "72a3879fd81c4c98bc2d65db2570e946"
format_stamp: "Formatted at 2026-08-09 08:56:22 on dist-test-slave-t3q3"
server_key: "e5db2f27cc4d6b29ceac72e3f6e4e938"
server_key_iv: "c893905a483e91c4daf78c36f85c3ef0"
server_key_version: "encryptionkey@0"
I20260809 08:56:22.467568 9889 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:56:22.468349 9919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:22.468518 9889 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:56:22.468612 9889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "72a3879fd81c4c98bc2d65db2570e946"
format_stamp: "Formatted at 2026-08-09 08:56:22 on dist-test-slave-t3q3"
server_key: "e5db2f27cc4d6b29ceac72e3f6e4e938"
server_key_iv: "c893905a483e91c4daf78c36f85c3ef0"
server_key_version: "encryptionkey@0"
I20260809 08:56:22.468693 9889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:22.471935 9889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:22.473745 9889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:22.473902 9889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:22.477952 9889 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46703
I20260809 08:56:22.477988 10004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46703 every 8 connection(s)
I20260809 08:56:22.478324 9889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:56:22.478588 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 9889
I20260809 08:56:22.478722 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:56:22.478972 8808 external_mini_cluster.cc:1428] Setting key cff1050de6674103e48658c9dccec312
I20260809 08:56:22.479600 10005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:22.481465 10013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:22.479546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:46469 (local address 127.0.0.1:46703)
0809 08:56:22.479727 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:22.479735 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:22.479758 (+ 23us) server_negotiation.cc:407] Connection header received
0809 08:56:22.480247 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:22.480255 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:22.480280 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:22.480392 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:22.481019 (+ 627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:22.481022 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:22.481022 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:56:22.481026 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:22.481063 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:56:22.481077 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:22.481135 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:22.481226 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 08:56:22.481279 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
I20260809 08:56:22.482473 8808 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:56:22.483031 10005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72a3879fd81c4c98bc2d65db2570e946: Bootstrap starting.
I20260809 08:56:22.483778 10005 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72a3879fd81c4c98bc2d65db2570e946: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:22.484166 10005 log.cc:824] T 00000000000000000000000000000000 P 72a3879fd81c4c98bc2d65db2570e946: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:22.484874 10005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72a3879fd81c4c98bc2d65db2570e946: No bootstrap required, opened a new log
I20260809 08:56:22.485178 10013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:22.484118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:56231 (local address 127.0.0.1:46703)
0809 08:56:22.484157 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:22.484159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:22.484255 (+ 96us) server_negotiation.cc:407] Connection header received
0809 08:56:22.484303 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:22.484304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:22.484308 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:22.484379 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:22.484948 (+ 569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:22.484950 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:22.484950 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:56:22.484955 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:22.484969 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:56:22.484973 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:22.485049 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:22.485111 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 08:56:22.485124 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 08:56:22.486855 10005 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72a3879fd81c4c98bc2d65db2570e946 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a3879fd81c4c98bc2d65db2570e946" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46703 } }
I20260809 08:56:22.486984 10005 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72a3879fd81c4c98bc2d65db2570e946 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:22.487006 10005 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72a3879fd81c4c98bc2d65db2570e946 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72a3879fd81c4c98bc2d65db2570e946, State: Initialized, Role: FOLLOWER
I20260809 08:56:22.487123 10005 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72a3879fd81c4c98bc2d65db2570e946 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a3879fd81c4c98bc2d65db2570e946" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46703 } }
I20260809 08:56:22.487214 10005 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72a3879fd81c4c98bc2d65db2570e946 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:56:22.487237 10005 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72a3879fd81c4c98bc2d65db2570e946 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:56:22.487267 10005 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72a3879fd81c4c98bc2d65db2570e946 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:22.487763 10005 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72a3879fd81c4c98bc2d65db2570e946 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a3879fd81c4c98bc2d65db2570e946" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46703 } }
I20260809 08:56:22.487854 10005 leader_election.cc:302] T 00000000000000000000000000000000 P 72a3879fd81c4c98bc2d65db2570e946 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72a3879fd81c4c98bc2d65db2570e946; no voters:
I20260809 08:56:22.487979 10005 leader_election.cc:288] T 00000000000000000000000000000000 P 72a3879fd81c4c98bc2d65db2570e946 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:56:22.488055 10028 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72a3879fd81c4c98bc2d65db2570e946 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:56:22.488247 10028 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72a3879fd81c4c98bc2d65db2570e946 [term 1 LEADER]: Becoming Leader. State: Replica: 72a3879fd81c4c98bc2d65db2570e946, State: Running, Role: LEADER
I20260809 08:56:22.488312 10005 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72a3879fd81c4c98bc2d65db2570e946 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:56:22.488345 10028 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72a3879fd81c4c98bc2d65db2570e946 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a3879fd81c4c98bc2d65db2570e946" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46703 } }
I20260809 08:56:22.488780 10029 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72a3879fd81c4c98bc2d65db2570e946 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72a3879fd81c4c98bc2d65db2570e946" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a3879fd81c4c98bc2d65db2570e946" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46703 } } }
I20260809 08:56:22.488790 10030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72a3879fd81c4c98bc2d65db2570e946 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72a3879fd81c4c98bc2d65db2570e946. Latest consensus state: current_term: 1 leader_uuid: "72a3879fd81c4c98bc2d65db2570e946" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72a3879fd81c4c98bc2d65db2570e946" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46703 } } }
I20260809 08:56:22.488983 10029 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72a3879fd81c4c98bc2d65db2570e946 [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:22.489002 10030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72a3879fd81c4c98bc2d65db2570e946 [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:22.489426 10038 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:56:22.489899 10038 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:56:22.491452 10038 catalog_manager.cc:1347] Generated new cluster ID: b028c5048944485c94721ac6a6f3f90c
I20260809 08:56:22.491515 10038 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:56:22.498685 10038 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:56:22.498878 10038 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:56:22.506501 10038 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72a3879fd81c4c98bc2d65db2570e946: Generated new TSK 0
I20260809 08:56:22.506748 10038 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:56:22.513732 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 9889
2026-08-09T08:56:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1541 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:56:22 dist-test-slave-t3q3 krb5kdc[10064](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:56:22 dist-test-slave-t3q3 krb5kdc[10064](info): set up 2 sockets
Aug 09 08:56:22 dist-test-slave-t3q3 krb5kdc[10064](info): commencing operation
krb5kdc: starting...
W20260809 08:56:23.761446 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.224s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:56:23 dist-test-slave-t3q3 krb5kdc[10064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:56:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:56:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 08:56:23.886969 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41201
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41201
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:56:23.972285 10089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:23.972496 10089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:23.972519 10089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:23.973706 10089 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:56:23.973752 10089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:23.973766 10089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:23.973779 10089 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:56:23.973830 10089 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:56:23.975310 10089 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41201
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10089
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:23.975576 10089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:23.975807 10089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:23.980823 10098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:23.980970 10089 server_base.cc:1034] running on GCE node
W20260809 08:56:23.980822 10096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:23.981072 10095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:23.981369 10089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:23.981676 10089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:23.982849 10089 hybrid_clock.cc:648] HybridClock initialized: now 1786265783982838 us; error 32 us; skew 500 ppm
Aug 09 08:56:23 dist-test-slave-t3q3 krb5kdc[10064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265783, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:56:23.985415 10089 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 08:56:23.985913 10089 webserver.cc:460] Webserver started at http://127.0.0.1:45015/ using document root <none> and password file <none>
I20260809 08:56:23.986153 10089 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:23.986223 10089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:23.986382 10089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:23.987471 10089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "70e9334c821d4df0ab69b50a5e29beba"
format_stamp: "Formatted at 2026-08-09 08:56:23 on dist-test-slave-t3q3"
server_key: "3ac59dea520163e05f6794eaa937b739"
server_key_iv: "ddd5df1d2b7be1600232aef2c63b0a63"
server_key_version: "encryptionkey@0"
I20260809 08:56:23.987852 10089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "70e9334c821d4df0ab69b50a5e29beba"
format_stamp: "Formatted at 2026-08-09 08:56:23 on dist-test-slave-t3q3"
server_key: "3ac59dea520163e05f6794eaa937b739"
server_key_iv: "ddd5df1d2b7be1600232aef2c63b0a63"
server_key_version: "encryptionkey@0"
I20260809 08:56:23.989284 10089 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:56:23.990098 10111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:23.990286 10089 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:56:23.990401 10089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "70e9334c821d4df0ab69b50a5e29beba"
format_stamp: "Formatted at 2026-08-09 08:56:23 on dist-test-slave-t3q3"
server_key: "3ac59dea520163e05f6794eaa937b739"
server_key_iv: "ddd5df1d2b7be1600232aef2c63b0a63"
server_key_version: "encryptionkey@0"
I20260809 08:56:23.990478 10089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:24.006287 10089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:24.008668 10089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:24.008837 10089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:24.012902 10089 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41201
I20260809 08:56:24.013000 10197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41201 every 8 connection(s)
I20260809 08:56:24.013455 10089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:56:24.014549 10201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:24.017009 10201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70e9334c821d4df0ab69b50a5e29beba: Bootstrap starting.
I20260809 08:56:24.017642 10201 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70e9334c821d4df0ab69b50a5e29beba: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:24.017939 10201 log.cc:824] T 00000000000000000000000000000000 P 70e9334c821d4df0ab69b50a5e29beba: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:24.018677 10201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70e9334c821d4df0ab69b50a5e29beba: No bootstrap required, opened a new log
I20260809 08:56:24.020418 10201 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70e9334c821d4df0ab69b50a5e29beba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70e9334c821d4df0ab69b50a5e29beba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41201 } }
I20260809 08:56:24.020560 10201 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70e9334c821d4df0ab69b50a5e29beba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:24.020583 10201 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70e9334c821d4df0ab69b50a5e29beba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70e9334c821d4df0ab69b50a5e29beba, State: Initialized, Role: FOLLOWER
I20260809 08:56:24.020750 10201 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70e9334c821d4df0ab69b50a5e29beba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70e9334c821d4df0ab69b50a5e29beba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41201 } }
I20260809 08:56:24.020815 10201 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70e9334c821d4df0ab69b50a5e29beba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:56:24.020836 10201 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70e9334c821d4df0ab69b50a5e29beba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:56:24.020900 10201 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70e9334c821d4df0ab69b50a5e29beba [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:24.021461 10201 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70e9334c821d4df0ab69b50a5e29beba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70e9334c821d4df0ab69b50a5e29beba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41201 } }
I20260809 08:56:24.021584 10201 leader_election.cc:302] T 00000000000000000000000000000000 P 70e9334c821d4df0ab69b50a5e29beba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70e9334c821d4df0ab69b50a5e29beba; no voters:
I20260809 08:56:24.021790 10201 leader_election.cc:288] T 00000000000000000000000000000000 P 70e9334c821d4df0ab69b50a5e29beba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:56:24.021842 10207 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70e9334c821d4df0ab69b50a5e29beba [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:56:24.022073 10201 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70e9334c821d4df0ab69b50a5e29beba [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:56:24.022074 10207 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70e9334c821d4df0ab69b50a5e29beba [term 1 LEADER]: Becoming Leader. State: Replica: 70e9334c821d4df0ab69b50a5e29beba, State: Running, Role: LEADER
I20260809 08:56:24.022272 10207 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70e9334c821d4df0ab69b50a5e29beba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70e9334c821d4df0ab69b50a5e29beba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41201 } }
I20260809 08:56:24.022425 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 10089
I20260809 08:56:24.022565 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:56:24.022770 8808 external_mini_cluster.cc:1428] Setting key 10efb7c0782b49ca754dbec0831d9d13
I20260809 08:56:24.022917 10209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70e9334c821d4df0ab69b50a5e29beba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70e9334c821d4df0ab69b50a5e29beba. Latest consensus state: current_term: 1 leader_uuid: "70e9334c821d4df0ab69b50a5e29beba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70e9334c821d4df0ab69b50a5e29beba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41201 } } }
I20260809 08:56:24.023000 10209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70e9334c821d4df0ab69b50a5e29beba [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:24.023209 10208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70e9334c821d4df0ab69b50a5e29beba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70e9334c821d4df0ab69b50a5e29beba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70e9334c821d4df0ab69b50a5e29beba" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41201 } } }
I20260809 08:56:24.023741 10208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70e9334c821d4df0ab69b50a5e29beba [sys.catalog]: This master's current role is: LEADER
W20260809 08:56:24.024286 10230 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 70e9334c821d4df0ab69b50a5e29beba: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:56:24.024356 10230 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:56:24.024417 10219 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:56:24.024596 10219 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:56:24.028113 10219 catalog_manager.cc:1347] Generated new cluster ID: 64e39f795d994738ad34ad48cc777a6b
I20260809 08:56:24.028196 10219 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 08:56:24 dist-test-slave-t3q3 krb5kdc[10064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265783, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 08:56:24.035547 10231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:24.023572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:58817 (local address 127.0.0.1:41201)
0809 08:56:24.023869 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:24.023881 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:24.023923 (+ 42us) server_negotiation.cc:407] Connection header received
0809 08:56:24.024211 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:24.024222 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:24.024319 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:24.024557 (+ 238us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:24.025380 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:24.026901 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:24.027697 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:24.028397 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:24.028517 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:24.031434 (+ 2917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:24.031439 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:24.031440 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:24.031454 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:24.033688 (+ 2234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:24.034163 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:24.034167 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:24.034167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:24.034305 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:24.034550 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:24.034551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:24.034552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:24.034898 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:24.035068 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:24.035183 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 08:56:24.035303 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":67,"threads_started":1}
I20260809 08:56:24.036736 8808 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:56:24.041479 10219 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:56:24.042192 10219 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:56:24.044804 10219 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70e9334c821d4df0ab69b50a5e29beba: Generated new TSK 0
I20260809 08:56:24.044970 10219 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:56:24.045403 10231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:24.038196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:57283 (local address 127.0.0.1:41201)
0809 08:56:24.038262 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:24.038264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:24.038380 (+ 116us) server_negotiation.cc:407] Connection header received
0809 08:56:24.038434 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:24.038435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:24.038445 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:24.038500 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:24.039136 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:24.039912 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:24.040977 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:24.041235 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:24.041328 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:24.042620 (+ 1292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:24.042621 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:24.042622 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:24.042629 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:24.044116 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:24.044487 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:24.044489 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:24.044489 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:24.044551 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:24.044793 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:24.044794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:24.044794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:24.044917 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:24.044993 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:24.045242 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 08:56:24.045308 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 08:56:24.047914 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 10089
W20260809 08:56:24.061616 10061 connection.cc:535] client connection to 127.0.0.1:41201 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41201): Connection reset by peer (error 104)
2026-08-09T08:56:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1538 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:56:24 dist-test-slave-t3q3 krb5kdc[10245](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:56:24 dist-test-slave-t3q3 krb5kdc[10245](info): set up 2 sockets
Aug 09 08:56:24 dist-test-slave-t3q3 krb5kdc[10245](info): commencing operation
krb5kdc: starting...
W20260809 08:56:25.309847 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.234s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:56:25 dist-test-slave-t3q3 krb5kdc[10245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:56:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:56:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 08:56:25.438781 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33469
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36383
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33469
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:56:25.522953 10273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:25.523187 10273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:25.523211 10273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:25.524533 10273 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:56:25.524580 10273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:25.524595 10273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:25.524608 10273 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:56:25.524621 10273 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:56:25.526206 10273 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36383
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33469
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10273
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:25.526486 10273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:25.526731 10273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:25.531699 10280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:25.531713 10273 server_base.cc:1034] running on GCE node
W20260809 08:56:25.531706 10285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:25.531761 10289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:25.532289 10273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:25.532584 10273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:25.533743 10273 hybrid_clock.cc:648] HybridClock initialized: now 1786265785533730 us; error 34 us; skew 500 ppm
Aug 09 08:56:25 dist-test-slave-t3q3 krb5kdc[10245](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265785, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:56:25.536320 10273 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 08:56:25.536777 10273 webserver.cc:460] Webserver started at http://127.0.0.1:39565/ using document root <none> and password file <none>
I20260809 08:56:25.537034 10273 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:25.537106 10273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:25.537259 10273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:25.538333 10273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "580e1c7dab214c949a8a836efc5a269f"
format_stamp: "Formatted at 2026-08-09 08:56:25 on dist-test-slave-t3q3"
server_key: "9204f155d5e12c74fcac817d3ae9dab6"
server_key_iv: "26c72bcbceb72dd5e281a1f71763ce28"
server_key_version: "encryptionkey@0"
I20260809 08:56:25.538779 10273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "580e1c7dab214c949a8a836efc5a269f"
format_stamp: "Formatted at 2026-08-09 08:56:25 on dist-test-slave-t3q3"
server_key: "9204f155d5e12c74fcac817d3ae9dab6"
server_key_iv: "26c72bcbceb72dd5e281a1f71763ce28"
server_key_version: "encryptionkey@0"
I20260809 08:56:25.540230 10273 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:56:25.541097 10302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:25.541282 10273 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:56:25.541405 10273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "580e1c7dab214c949a8a836efc5a269f"
format_stamp: "Formatted at 2026-08-09 08:56:25 on dist-test-slave-t3q3"
server_key: "9204f155d5e12c74fcac817d3ae9dab6"
server_key_iv: "26c72bcbceb72dd5e281a1f71763ce28"
server_key_version: "encryptionkey@0"
I20260809 08:56:25.541510 10273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:25.568902 10273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:25.571096 10273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:25.571283 10273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:25.575908 10273 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33469
I20260809 08:56:25.575951 10416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33469 every 8 connection(s)
I20260809 08:56:25.576407 10273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:56:25.577349 10418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:25.579627 10418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 580e1c7dab214c949a8a836efc5a269f: Bootstrap starting.
I20260809 08:56:25.580272 10418 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 580e1c7dab214c949a8a836efc5a269f: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:25.580524 10418 log.cc:824] T 00000000000000000000000000000000 P 580e1c7dab214c949a8a836efc5a269f: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:25.581230 10418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 580e1c7dab214c949a8a836efc5a269f: No bootstrap required, opened a new log
I20260809 08:56:25.582789 10418 raft_consensus.cc:348] T 00000000000000000000000000000000 P 580e1c7dab214c949a8a836efc5a269f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "580e1c7dab214c949a8a836efc5a269f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33469 } }
I20260809 08:56:25.582932 10418 raft_consensus.cc:374] T 00000000000000000000000000000000 P 580e1c7dab214c949a8a836efc5a269f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:25.582957 10418 raft_consensus.cc:729] T 00000000000000000000000000000000 P 580e1c7dab214c949a8a836efc5a269f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 580e1c7dab214c949a8a836efc5a269f, State: Initialized, Role: FOLLOWER
I20260809 08:56:25.583069 10418 consensus_queue.cc:260] T 00000000000000000000000000000000 P 580e1c7dab214c949a8a836efc5a269f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "580e1c7dab214c949a8a836efc5a269f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33469 } }
I20260809 08:56:25.583122 10418 raft_consensus.cc:388] T 00000000000000000000000000000000 P 580e1c7dab214c949a8a836efc5a269f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:56:25.583143 10418 raft_consensus.cc:482] T 00000000000000000000000000000000 P 580e1c7dab214c949a8a836efc5a269f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:56:25.583169 10418 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 580e1c7dab214c949a8a836efc5a269f [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:25.583657 10418 raft_consensus.cc:504] T 00000000000000000000000000000000 P 580e1c7dab214c949a8a836efc5a269f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "580e1c7dab214c949a8a836efc5a269f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33469 } }
I20260809 08:56:25.583742 10418 leader_election.cc:302] T 00000000000000000000000000000000 P 580e1c7dab214c949a8a836efc5a269f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 580e1c7dab214c949a8a836efc5a269f; no voters:
I20260809 08:56:25.583985 10423 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 580e1c7dab214c949a8a836efc5a269f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:56:25.583987 10418 leader_election.cc:288] T 00000000000000000000000000000000 P 580e1c7dab214c949a8a836efc5a269f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:56:25.584268 10423 raft_consensus.cc:686] T 00000000000000000000000000000000 P 580e1c7dab214c949a8a836efc5a269f [term 1 LEADER]: Becoming Leader. State: Replica: 580e1c7dab214c949a8a836efc5a269f, State: Running, Role: LEADER
I20260809 08:56:25.584440 10423 consensus_queue.cc:237] T 00000000000000000000000000000000 P 580e1c7dab214c949a8a836efc5a269f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "580e1c7dab214c949a8a836efc5a269f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33469 } }
I20260809 08:56:25.584607 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 10273
I20260809 08:56:25.584733 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:56:25.584856 10423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 580e1c7dab214c949a8a836efc5a269f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "580e1c7dab214c949a8a836efc5a269f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "580e1c7dab214c949a8a836efc5a269f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33469 } } }
I20260809 08:56:25.584931 8808 external_mini_cluster.cc:1428] Setting key b82edb7fffcb065ed686ab5710c3f09c
I20260809 08:56:25.584954 10423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 580e1c7dab214c949a8a836efc5a269f [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:25.585125 10424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 580e1c7dab214c949a8a836efc5a269f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 580e1c7dab214c949a8a836efc5a269f. Latest consensus state: current_term: 1 leader_uuid: "580e1c7dab214c949a8a836efc5a269f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "580e1c7dab214c949a8a836efc5a269f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33469 } } }
I20260809 08:56:25.585181 10424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 580e1c7dab214c949a8a836efc5a269f [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:25.585587 10418 sys_catalog.cc:564] T 00000000000000000000000000000000 P 580e1c7dab214c949a8a836efc5a269f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:56:25.585639 10431 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:56:25.586197 10431 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:56:25.588104 10431 catalog_manager.cc:1347] Generated new cluster ID: 9d587e1f63034b8fa1b70c7cb4ff1add
I20260809 08:56:25.588155 10431 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:56:25.592697 10434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:25.585841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:52839 (local address 127.0.0.1:33469)
0809 08:56:25.586024 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:25.586035 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:25.587200 (+ 1165us) server_negotiation.cc:407] Connection header received
0809 08:56:25.587452 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:25.587465 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:25.587512 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:25.587622 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:25.587895 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:25.589313 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:25.590471 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:25.591086 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:25.591196 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:25.591970 (+ 774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:25.591975 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:25.591975 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:56:25.592203 (+ 228us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:25.592249 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:56:25.592264 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:25.592346 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:25.592464 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 08:56:25.592490 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":71,"threads_started":1}
I20260809 08:56:25.593842 8808 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:56:25.604844 10434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:25.601097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:46895 (local address 127.0.0.1:33469)
0809 08:56:25.601147 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:25.601149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:25.601312 (+ 163us) server_negotiation.cc:407] Connection header received
0809 08:56:25.601359 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:25.601360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:25.601373 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:25.601429 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:25.601578 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:25.602368 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:25.603376 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:25.603695 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:25.603772 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:25.604557 (+ 785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:25.604558 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:25.604559 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:56:25.604564 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:25.604580 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:56:25.604585 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:25.604651 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:25.604739 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 08:56:25.604753 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":17}
I20260809 08:56:25.614976 10431 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:56:25.615573 10431 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:56:25.622574 10431 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 580e1c7dab214c949a8a836efc5a269f: Generated new TSK 0
I20260809 08:56:25.622779 10431 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:56:25.625716 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 10273
W20260809 08:56:25.646253 10241 connection.cc:535] client connection to 127.0.0.1:33469 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33469): Connection reset by peer (error 104)
2026-08-09T08:56:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1584 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:56:25 dist-test-slave-t3q3 krb5kdc[10471](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:56:25 dist-test-slave-t3q3 krb5kdc[10471](info): set up 2 sockets
Aug 09 08:56:25 dist-test-slave-t3q3 krb5kdc[10471](info): commencing operation
krb5kdc: starting...
W20260809 08:56:26.887670 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.227s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:56:26 dist-test-slave-t3q3 krb5kdc[10471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265786, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:56:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:56:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 08:56:27.014150 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35029
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35029
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:56:27.095254 10496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:27.095455 10496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:27.095479 10496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:27.096639 10496 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:56:27.096683 10496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:27.096698 10496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:27.096711 10496 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:56:27.096766 10496 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:56:27.098222 10496 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35029
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265519942966-8808-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:27.098474 10496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:27.098701 10496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:27.103466 10509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:27.103483 10505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:27.103473 10504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:27.103801 10496 server_base.cc:1034] running on GCE node
I20260809 08:56:27.104050 10496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:27.104382 10496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:27.105566 10496 hybrid_clock.cc:648] HybridClock initialized: now 1786265787105548 us; error 42 us; skew 500 ppm
Aug 09 08:56:27 dist-test-slave-t3q3 krb5kdc[10471](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265787, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:56:27.108129 10496 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 08:56:27.108525 10496 webserver.cc:460] Webserver started at http://127.0.0.1:42555/ using document root <none> and password file <none>
I20260809 08:56:27.108775 10496 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:27.108824 10496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:27.108912 10496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:27.109920 10496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "023b4b98b74f4324bb6d32405972762a"
format_stamp: "Formatted at 2026-08-09 08:56:27 on dist-test-slave-t3q3"
server_key: "2b12813d42db7e9815bf94fdf71debe8"
server_key_iv: "4fe53dcaf70781f1d443f53408c8a8fd"
server_key_version: "encryptionkey@0"
I20260809 08:56:27.110224 10496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "023b4b98b74f4324bb6d32405972762a"
format_stamp: "Formatted at 2026-08-09 08:56:27 on dist-test-slave-t3q3"
server_key: "2b12813d42db7e9815bf94fdf71debe8"
server_key_iv: "4fe53dcaf70781f1d443f53408c8a8fd"
server_key_version: "encryptionkey@0"
I20260809 08:56:27.111485 10496 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 08:56:27.112216 10520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:27.112386 10496 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:56:27.112476 10496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "023b4b98b74f4324bb6d32405972762a"
format_stamp: "Formatted at 2026-08-09 08:56:27 on dist-test-slave-t3q3"
server_key: "2b12813d42db7e9815bf94fdf71debe8"
server_key_iv: "4fe53dcaf70781f1d443f53408c8a8fd"
server_key_version: "encryptionkey@0"
I20260809 08:56:27.112550 10496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:27.115698 10496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:27.117559 10496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:27.117727 10496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:27.121624 10496 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35029
I20260809 08:56:27.121659 10604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35029 every 8 connection(s)
I20260809 08:56:27.121984 10496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:56:27.122928 10607 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:27.125198 10607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 023b4b98b74f4324bb6d32405972762a: Bootstrap starting.
I20260809 08:56:27.125773 10607 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 023b4b98b74f4324bb6d32405972762a: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:27.126060 10607 log.cc:824] T 00000000000000000000000000000000 P 023b4b98b74f4324bb6d32405972762a: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:27.126734 10607 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 023b4b98b74f4324bb6d32405972762a: No bootstrap required, opened a new log
I20260809 08:56:27.128369 10607 raft_consensus.cc:348] T 00000000000000000000000000000000 P 023b4b98b74f4324bb6d32405972762a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "023b4b98b74f4324bb6d32405972762a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35029 } }
I20260809 08:56:27.128504 10607 raft_consensus.cc:374] T 00000000000000000000000000000000 P 023b4b98b74f4324bb6d32405972762a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:27.128548 10607 raft_consensus.cc:729] T 00000000000000000000000000000000 P 023b4b98b74f4324bb6d32405972762a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 023b4b98b74f4324bb6d32405972762a, State: Initialized, Role: FOLLOWER
I20260809 08:56:27.128712 10607 consensus_queue.cc:260] T 00000000000000000000000000000000 P 023b4b98b74f4324bb6d32405972762a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "023b4b98b74f4324bb6d32405972762a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35029 } }
I20260809 08:56:27.128777 10607 raft_consensus.cc:388] T 00000000000000000000000000000000 P 023b4b98b74f4324bb6d32405972762a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:56:27.128816 10607 raft_consensus.cc:482] T 00000000000000000000000000000000 P 023b4b98b74f4324bb6d32405972762a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:56:27.128866 10607 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 023b4b98b74f4324bb6d32405972762a [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:27.129069 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 10496
I20260809 08:56:27.129217 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:56:27.129397 8808 external_mini_cluster.cc:1428] Setting key 0138ab1768f154b23f95bed7dd37c1c2
I20260809 08:56:27.129513 10607 raft_consensus.cc:504] T 00000000000000000000000000000000 P 023b4b98b74f4324bb6d32405972762a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "023b4b98b74f4324bb6d32405972762a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35029 } }
I20260809 08:56:27.129659 10607 leader_election.cc:302] T 00000000000000000000000000000000 P 023b4b98b74f4324bb6d32405972762a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 023b4b98b74f4324bb6d32405972762a; no voters:
I20260809 08:56:27.129881 10607 leader_election.cc:288] T 00000000000000000000000000000000 P 023b4b98b74f4324bb6d32405972762a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:56:27.129947 10610 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 023b4b98b74f4324bb6d32405972762a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:56:27.130065 10610 raft_consensus.cc:686] T 00000000000000000000000000000000 P 023b4b98b74f4324bb6d32405972762a [term 1 LEADER]: Becoming Leader. State: Replica: 023b4b98b74f4324bb6d32405972762a, State: Running, Role: LEADER
I20260809 08:56:27.130152 10610 consensus_queue.cc:237] T 00000000000000000000000000000000 P 023b4b98b74f4324bb6d32405972762a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "023b4b98b74f4324bb6d32405972762a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35029 } }
I20260809 08:56:27.130227 10607 sys_catalog.cc:564] T 00000000000000000000000000000000 P 023b4b98b74f4324bb6d32405972762a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:56:27.130458 10610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 023b4b98b74f4324bb6d32405972762a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 023b4b98b74f4324bb6d32405972762a. Latest consensus state: current_term: 1 leader_uuid: "023b4b98b74f4324bb6d32405972762a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "023b4b98b74f4324bb6d32405972762a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35029 } } }
I20260809 08:56:27.130471 10616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 023b4b98b74f4324bb6d32405972762a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "023b4b98b74f4324bb6d32405972762a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "023b4b98b74f4324bb6d32405972762a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35029 } } }
I20260809 08:56:27.130632 10610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 023b4b98b74f4324bb6d32405972762a [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:27.130848 10616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 023b4b98b74f4324bb6d32405972762a [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:27.131925 10619 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:56:27.132452 10613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:27.129938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:57657 (local address 127.0.0.1:35029)
0809 08:56:27.130615 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:27.130624 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:27.130667 (+ 43us) server_negotiation.cc:407] Connection header received
0809 08:56:27.130790 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:27.130798 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:27.130830 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:27.130967 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:27.131968 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:27.131972 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:27.131973 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:56:27.131978 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:27.132030 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:56:27.132045 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:27.132137 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:27.132214 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 08:56:27.132260 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":88,"threads_started":1}
I20260809 08:56:27.132658 10619 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:56:27.134912 8808 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:56:27.135573 10619 catalog_manager.cc:1347] Generated new cluster ID: cdb6049b9246484885699fae2463717b
I20260809 08:56:27.135618 10619 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:56:27.137776 10613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:27.136634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:42063 (local address 127.0.0.1:35029)
0809 08:56:27.136670 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:27.136672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:27.136875 (+ 203us) server_negotiation.cc:407] Connection header received
0809 08:56:27.136950 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:27.136951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:27.136957 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:27.137012 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:27.137564 (+ 552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:27.137566 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:27.137566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:56:27.137571 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:27.137585 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:56:27.137589 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:27.137637 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:27.137689 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 08:56:27.137705 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 08:56:27.148479 10619 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:56:27.148640 10619 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:56:27.162039 10619 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 023b4b98b74f4324bb6d32405972762a: Generated new TSK 0
I20260809 08:56:27.162254 10619 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:56:27.164827 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 10496
2026-08-09T08:56:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1532 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:56:27 dist-test-slave-t3q3 krb5kdc[10656](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:56:27 dist-test-slave-t3q3 krb5kdc[10656](info): set up 2 sockets
Aug 09 08:56:27 dist-test-slave-t3q3 krb5kdc[10656](info): commencing operation
krb5kdc: starting...
W20260809 08:56:28.428481 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:56:28 dist-test-slave-t3q3 krb5kdc[10656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265788, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:56:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:56:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
I20260809 08:56:28.549024 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:45731
--webserver_interface=127.8.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:33653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:45731
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:56:28.630393 10679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:28.630585 10679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:28.630607 10679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:28.631749 10679 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:56:28.631793 10679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:28.631806 10679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:28.631819 10679 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:56:28.631999 10679 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:56:28.633788 10679 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:33653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:45731
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:45731
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:28.634038 10679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:28.634277 10679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:28.639416 10692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:28.639469 10679 server_base.cc:1034] running on GCE node
W20260809 08:56:28.639386 10689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:28.639390 10688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:28.640110 10679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:28.640388 10679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:28.641562 10679 hybrid_clock.cc:648] HybridClock initialized: now 1786265788641542 us; error 33 us; skew 500 ppm
Aug 09 08:56:28 dist-test-slave-t3q3 krb5kdc[10656](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265788, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:56:28.644255 10679 init.cc:377] Logged in from keytab as kudu/127.8.154.62@KRBTEST.COM (short username kudu)
I20260809 08:56:28.644649 10679 webserver.cc:460] Webserver started at http://127.8.154.62:38859/ using document root <none> and password file <none>
I20260809 08:56:28.644848 10679 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:28.644888 10679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:28.644975 10679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:28.645982 10679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "4e66b18c1b934f0ab65b657481815dc0"
format_stamp: "Formatted at 2026-08-09 08:56:28 on dist-test-slave-t3q3"
server_key: "51e8f9eb0d7ce113370749d4379c2fb3"
server_key_iv: "37401766f23ce1245508fec9fa1660c2"
server_key_version: "encryptionkey@0"
I20260809 08:56:28.646283 10679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "4e66b18c1b934f0ab65b657481815dc0"
format_stamp: "Formatted at 2026-08-09 08:56:28 on dist-test-slave-t3q3"
server_key: "51e8f9eb0d7ce113370749d4379c2fb3"
server_key_iv: "37401766f23ce1245508fec9fa1660c2"
server_key_version: "encryptionkey@0"
I20260809 08:56:28.647536 10679 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260809 08:56:28.648265 10702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:28.648423 10679 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:28.648516 10679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "4e66b18c1b934f0ab65b657481815dc0"
format_stamp: "Formatted at 2026-08-09 08:56:28 on dist-test-slave-t3q3"
server_key: "51e8f9eb0d7ce113370749d4379c2fb3"
server_key_iv: "37401766f23ce1245508fec9fa1660c2"
server_key_version: "encryptionkey@0"
I20260809 08:56:28.648571 10679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:28.664604 10679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:28.666426 10679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:28.666540 10679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:28.670553 10679 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:45731
I20260809 08:56:28.670606 10789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:45731 every 8 connection(s)
I20260809 08:56:28.670950 10679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:56:28.671931 10790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:28.673801 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 10679
I20260809 08:56:28.673906 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:56:28.674130 8808 external_mini_cluster.cc:1428] Setting key 7bc2d3c12756cb391d2d63fe1db60599
I20260809 08:56:28.674508 10790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e66b18c1b934f0ab65b657481815dc0: Bootstrap starting.
I20260809 08:56:28.675179 10790 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e66b18c1b934f0ab65b657481815dc0: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:28.675474 10790 log.cc:824] T 00000000000000000000000000000000 P 4e66b18c1b934f0ab65b657481815dc0: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:28.676263 10790 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e66b18c1b934f0ab65b657481815dc0: No bootstrap required, opened a new log
I20260809 08:56:28.678104 10790 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e66b18c1b934f0ab65b657481815dc0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e66b18c1b934f0ab65b657481815dc0" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45731 } }
I20260809 08:56:28.678268 10790 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e66b18c1b934f0ab65b657481815dc0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:28.678319 10790 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e66b18c1b934f0ab65b657481815dc0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e66b18c1b934f0ab65b657481815dc0, State: Initialized, Role: FOLLOWER
I20260809 08:56:28.678476 10790 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e66b18c1b934f0ab65b657481815dc0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e66b18c1b934f0ab65b657481815dc0" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45731 } }
I20260809 08:56:28.678576 10790 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e66b18c1b934f0ab65b657481815dc0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:56:28.678629 10790 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e66b18c1b934f0ab65b657481815dc0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:56:28.678680 10790 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e66b18c1b934f0ab65b657481815dc0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:28.679318 10790 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e66b18c1b934f0ab65b657481815dc0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e66b18c1b934f0ab65b657481815dc0" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45731 } }
I20260809 08:56:28.679435 10790 leader_election.cc:302] T 00000000000000000000000000000000 P 4e66b18c1b934f0ab65b657481815dc0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e66b18c1b934f0ab65b657481815dc0; no voters:
I20260809 08:56:28.679651 10790 leader_election.cc:288] T 00000000000000000000000000000000 P 4e66b18c1b934f0ab65b657481815dc0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:56:28.679672 10799 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e66b18c1b934f0ab65b657481815dc0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:56:28.679796 10799 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e66b18c1b934f0ab65b657481815dc0 [term 1 LEADER]: Becoming Leader. State: Replica: 4e66b18c1b934f0ab65b657481815dc0, State: Running, Role: LEADER
I20260809 08:56:28.679960 10790 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e66b18c1b934f0ab65b657481815dc0 [sys.catalog]: configured and running, proceeding with master startup.
Aug 09 08:56:28 dist-test-slave-t3q3 krb5kdc[10656](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265788, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:56:28.679944 10799 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e66b18c1b934f0ab65b657481815dc0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e66b18c1b934f0ab65b657481815dc0" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45731 } }
I20260809 08:56:28.681641 10799 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e66b18c1b934f0ab65b657481815dc0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e66b18c1b934f0ab65b657481815dc0. Latest consensus state: current_term: 1 leader_uuid: "4e66b18c1b934f0ab65b657481815dc0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e66b18c1b934f0ab65b657481815dc0" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45731 } } }
I20260809 08:56:28.681792 10799 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e66b18c1b934f0ab65b657481815dc0 [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:28.681821 10800 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e66b18c1b934f0ab65b657481815dc0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e66b18c1b934f0ab65b657481815dc0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e66b18c1b934f0ab65b657481815dc0" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45731 } } }
I20260809 08:56:28.681933 10800 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e66b18c1b934f0ab65b657481815dc0 [sys.catalog]: This master's current role is: LEADER
W20260809 08:56:28.682655 10820 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4e66b18c1b934f0ab65b657481815dc0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:56:28.682744 10820 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:56:28.682809 10816 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:56:28.682981 10816 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:56:28.685103 10816 catalog_manager.cc:1347] Generated new cluster ID: c7266f55028548bfbb280ada08358c7b
I20260809 08:56:28.685155 10816 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:56:28.685999 10796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:28.674680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35041 (local address 127.8.154.62:45731)
0809 08:56:28.674876 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:28.674886 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:28.674918 (+ 32us) server_negotiation.cc:407] Connection header received
0809 08:56:28.675403 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:28.675410 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:28.675458 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:28.675609 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:28.676334 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:28.677290 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:28.678063 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:28.678574 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:28.678705 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:28.681814 (+ 3109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:28.681818 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:28.681819 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:28.681831 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:28.683833 (+ 2002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:28.684331 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:28.684334 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:28.684334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:28.684436 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:28.684727 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:28.684729 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:28.684729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:28.685040 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:28.685207 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:28.685670 (+ 463us) server_negotiation.cc:299] Negotiation successful
0809 08:56:28.685776 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":85,"threads_started":1}
I20260809 08:56:28.686973 8808 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:56:28.690485 10816 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:56:28.690959 10816 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:56:28.694866 10816 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e66b18c1b934f0ab65b657481815dc0: Generated new TSK 0
I20260809 08:56:28.695086 10816 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:56:28.698727 10796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:28.688475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53655 (local address 127.8.154.62:45731)
0809 08:56:28.688507 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:28.688509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:28.690190 (+ 1681us) server_negotiation.cc:407] Connection header received
0809 08:56:28.690387 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:28.690388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:28.690398 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:28.690450 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:28.692716 (+ 2266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:28.693549 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:28.694521 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:28.694854 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:28.694927 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:28.696130 (+ 1203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:28.696131 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:28.696132 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:28.696138 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:28.697608 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:28.697973 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:28.697974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:28.697975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:28.698035 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:28.698248 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:28.698249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:28.698249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:28.698375 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:28.698472 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:28.698570 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 08:56:28.698635 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":17}
I20260809 08:56:28.700860 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 10679
W20260809 08:56:28.718452 10654 connection.cc:535] client connection to 127.8.154.62:45731 recv error: Network error: failed to read from TLS socket (remote: 127.8.154.62:45731): Connection reset by peer (error 104)
2026-08-09T08:56:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1541 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:56:28 dist-test-slave-t3q3 krb5kdc[10836](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:56:28 dist-test-slave-t3q3 krb5kdc[10836](info): set up 2 sockets
Aug 09 08:56:28 dist-test-slave-t3q3 krb5kdc[10836](info): commencing operation
krb5kdc: starting...
W20260809 08:56:29.966933 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.233s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:56:30 dist-test-slave-t3q3 krb5kdc[10836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:56:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:56:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
I20260809 08:56:30.091252 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:45917
--webserver_interface=127.8.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:35065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:45917
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:56:30.175894 10856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:30.176142 10856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:30.176184 10856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:30.177330 10856 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:56:30.177397 10856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:30.177423 10856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:30.177452 10856 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:56:30.177476 10856 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:56:30.178937 10856 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:35065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:45917
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:45917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:30.179219 10856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:30.179442 10856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:30.184330 10866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:30.184355 10869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:30.184355 10865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:30.184433 10856 server_base.cc:1034] running on GCE node
I20260809 08:56:30.184774 10856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:30.185066 10856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:30.186228 10856 hybrid_clock.cc:648] HybridClock initialized: now 1786265790186216 us; error 45 us; skew 500 ppm
Aug 09 08:56:30 dist-test-slave-t3q3 krb5kdc[10836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265790, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:56:30.188634 10856 init.cc:377] Logged in from keytab as kudu/127.8.154.62@KRBTEST.COM (short username kudu)
I20260809 08:56:30.189128 10856 webserver.cc:460] Webserver started at http://127.8.154.62:40285/ using document root <none> and password file <none>
I20260809 08:56:30.189365 10856 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:30.189433 10856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:30.189589 10856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:30.190580 10856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "0af84d0263ac4219808ec904cfccb910"
format_stamp: "Formatted at 2026-08-09 08:56:30 on dist-test-slave-t3q3"
server_key: "efc1e3c58ad25b3680e73b1bc4c6bb56"
server_key_iv: "18461148317a486b938631a440fe1236"
server_key_version: "encryptionkey@0"
I20260809 08:56:30.190938 10856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "0af84d0263ac4219808ec904cfccb910"
format_stamp: "Formatted at 2026-08-09 08:56:30 on dist-test-slave-t3q3"
server_key: "efc1e3c58ad25b3680e73b1bc4c6bb56"
server_key_iv: "18461148317a486b938631a440fe1236"
server_key_version: "encryptionkey@0"
I20260809 08:56:30.192327 10856 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 08:56:30.193049 10878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:30.193225 10856 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:30.193333 10856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "0af84d0263ac4219808ec904cfccb910"
format_stamp: "Formatted at 2026-08-09 08:56:30 on dist-test-slave-t3q3"
server_key: "efc1e3c58ad25b3680e73b1bc4c6bb56"
server_key_iv: "18461148317a486b938631a440fe1236"
server_key_version: "encryptionkey@0"
I20260809 08:56:30.193408 10856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:30.203512 10856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:30.205453 10856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:30.205624 10856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:30.210032 10856 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:45917
I20260809 08:56:30.210064 10967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:45917 every 8 connection(s)
I20260809 08:56:30.210451 10856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:56:30.211328 10969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:30.213869 10969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0af84d0263ac4219808ec904cfccb910: Bootstrap starting.
I20260809 08:56:30.214460 10969 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0af84d0263ac4219808ec904cfccb910: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:30.214751 10969 log.cc:824] T 00000000000000000000000000000000 P 0af84d0263ac4219808ec904cfccb910: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:30.215476 10969 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0af84d0263ac4219808ec904cfccb910: No bootstrap required, opened a new log
I20260809 08:56:30.215983 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 10856
I20260809 08:56:30.216187 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:56:30.216382 8808 external_mini_cluster.cc:1428] Setting key c5ebc9efa0f8711caacd1131eeec917c
I20260809 08:56:30.217777 10969 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0af84d0263ac4219808ec904cfccb910 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af84d0263ac4219808ec904cfccb910" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45917 } }
I20260809 08:56:30.217986 10969 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0af84d0263ac4219808ec904cfccb910 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:30.218051 10969 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0af84d0263ac4219808ec904cfccb910 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0af84d0263ac4219808ec904cfccb910, State: Initialized, Role: FOLLOWER
I20260809 08:56:30.218211 10969 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0af84d0263ac4219808ec904cfccb910 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af84d0263ac4219808ec904cfccb910" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45917 } }
I20260809 08:56:30.218304 10969 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0af84d0263ac4219808ec904cfccb910 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:56:30.218338 10969 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0af84d0263ac4219808ec904cfccb910 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:56:30.218384 10969 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0af84d0263ac4219808ec904cfccb910 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:30.219123 10969 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0af84d0263ac4219808ec904cfccb910 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af84d0263ac4219808ec904cfccb910" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45917 } }
I20260809 08:56:30.219240 10969 leader_election.cc:302] T 00000000000000000000000000000000 P 0af84d0263ac4219808ec904cfccb910 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0af84d0263ac4219808ec904cfccb910; no voters:
I20260809 08:56:30.219389 10969 leader_election.cc:288] T 00000000000000000000000000000000 P 0af84d0263ac4219808ec904cfccb910 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:56:30.219453 10978 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0af84d0263ac4219808ec904cfccb910 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:56:30.219617 10969 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0af84d0263ac4219808ec904cfccb910 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:56:30.219659 10978 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0af84d0263ac4219808ec904cfccb910 [term 1 LEADER]: Becoming Leader. State: Replica: 0af84d0263ac4219808ec904cfccb910, State: Running, Role: LEADER
I20260809 08:56:30.219863 10978 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0af84d0263ac4219808ec904cfccb910 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af84d0263ac4219808ec904cfccb910" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45917 } }
I20260809 08:56:30.220275 10979 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0af84d0263ac4219808ec904cfccb910 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0af84d0263ac4219808ec904cfccb910" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af84d0263ac4219808ec904cfccb910" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45917 } } }
I20260809 08:56:30.220357 10979 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0af84d0263ac4219808ec904cfccb910 [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:30.220525 10981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0af84d0263ac4219808ec904cfccb910 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0af84d0263ac4219808ec904cfccb910. Latest consensus state: current_term: 1 leader_uuid: "0af84d0263ac4219808ec904cfccb910" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af84d0263ac4219808ec904cfccb910" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 45917 } } }
I20260809 08:56:30.220594 10981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0af84d0263ac4219808ec904cfccb910 [sys.catalog]: This master's current role is: LEADER
W20260809 08:56:30.221485 11004 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0af84d0263ac4219808ec904cfccb910: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:56:30.221573 11004 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:56:30.221658 10991 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:56:30.221853 10991 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:56:30.223474 10991 catalog_manager.cc:1347] Generated new cluster ID: f8cd5952d82547cf81b8c634c777de36
I20260809 08:56:30.223506 10977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:30.217044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38985 (local address 127.8.154.62:45917)
0809 08:56:30.217252 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:30.217262 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:30.217298 (+ 36us) server_negotiation.cc:407] Connection header received
0809 08:56:30.217732 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:30.217742 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:30.217807 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:30.217956 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:30.218312 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:30.220090 (+ 1778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:30.221365 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:30.221866 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:30.221988 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:30.222752 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:30.222755 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:30.222756 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:56:30.222987 (+ 231us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:30.223031 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:56:30.223046 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:30.223137 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:30.223219 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 08:56:30.223347 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":107,"threads_started":1}
I20260809 08:56:30.223521 10991 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:56:30.224785 8808 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:56:30.229823 10977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:30.226360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35215 (local address 127.8.154.62:45917)
0809 08:56:30.226390 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:30.226391 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:30.226399 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:56:30.226477 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:30.226477 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:30.226485 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:30.226533 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:30.226662 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:30.227366 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:30.228480 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:30.228792 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:30.228853 (+ 61us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:30.229556 (+ 703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:30.229557 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:30.229557 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:56:30.229561 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:30.229575 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:56:30.229579 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:30.229635 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:30.229743 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 08:56:30.229755 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 08:56:30.240856 10991 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:56:30.242260 10991 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:56:30.246529 10991 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0af84d0263ac4219808ec904cfccb910: Generated new TSK 0
I20260809 08:56:30.246711 10991 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:56:30.251123 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 10856
W20260809 08:56:30.263355 10834 connection.cc:535] client connection to 127.8.154.62:45917 recv error: Network error: failed to read from TLS socket (remote: 127.8.154.62:45917): Connection reset by peer (error 104)
2026-08-09T08:56:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1543 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:56:30 dist-test-slave-t3q3 krb5kdc[11016](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:56:30 dist-test-slave-t3q3 krb5kdc[11016](info): set up 2 sockets
Aug 09 08:56:30 dist-test-slave-t3q3 krb5kdc[11016](info): commencing operation
krb5kdc: starting...
W20260809 08:56:31.502411 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.225s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:56:31 dist-test-slave-t3q3 krb5kdc[11016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265791, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:56:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:56:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
I20260809 08:56:31.626467 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:43639
--webserver_interface=127.8.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:40545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:43639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:56:31.709178 11033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:31.709415 11033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:31.709455 11033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:31.710572 11033 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:56:31.710645 11033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:31.710671 11033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:31.710700 11033 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:56:31.710733 11033 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:56:31.712278 11033 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:40545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:43639
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:43639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:31.712549 11033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:31.712782 11033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:31.717764 11041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:31.717778 11040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:31.717778 11043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:31.717918 11033 server_base.cc:1034] running on GCE node
I20260809 08:56:31.718286 11033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:31.718565 11033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:31.719743 11033 hybrid_clock.cc:648] HybridClock initialized: now 1786265791719728 us; error 46 us; skew 500 ppm
Aug 09 08:56:31 dist-test-slave-t3q3 krb5kdc[11016](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265791, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:56:31.722290 11033 init.cc:377] Logged in from keytab as kudu/127.8.154.62@KRBTEST.COM (short username kudu)
I20260809 08:56:31.722795 11033 webserver.cc:460] Webserver started at http://127.8.154.62:33437/ using document root <none> and password file <none>
I20260809 08:56:31.723116 11033 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:31.723219 11033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:31.723390 11033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:31.724557 11033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "22b4c1d9aeb74b0b8e99ffe5e616be63"
format_stamp: "Formatted at 2026-08-09 08:56:31 on dist-test-slave-t3q3"
server_key: "84f54219abeaad586eed8f63af8418de"
server_key_iv: "e5026329d534e5dc565f663cf0e7ca67"
server_key_version: "encryptionkey@0"
I20260809 08:56:31.724962 11033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "22b4c1d9aeb74b0b8e99ffe5e616be63"
format_stamp: "Formatted at 2026-08-09 08:56:31 on dist-test-slave-t3q3"
server_key: "84f54219abeaad586eed8f63af8418de"
server_key_iv: "e5026329d534e5dc565f663cf0e7ca67"
server_key_version: "encryptionkey@0"
I20260809 08:56:31.726331 11033 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:56:31.727134 11054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:31.727425 11033 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:56:31.727535 11033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "22b4c1d9aeb74b0b8e99ffe5e616be63"
format_stamp: "Formatted at 2026-08-09 08:56:31 on dist-test-slave-t3q3"
server_key: "84f54219abeaad586eed8f63af8418de"
server_key_iv: "e5026329d534e5dc565f663cf0e7ca67"
server_key_version: "encryptionkey@0"
I20260809 08:56:31.727612 11033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:31.731066 11033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:31.733125 11033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:31.733283 11033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:31.740375 11033 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:43639
I20260809 08:56:31.740612 11151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:43639 every 8 connection(s)
I20260809 08:56:31.740932 11033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:56:31.741562 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 11033
I20260809 08:56:31.741693 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:56:31.741923 8808 external_mini_cluster.cc:1428] Setting key aedf683381c0877244c7a54985ae32f4
I20260809 08:56:31.742249 11154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:31.746537 11154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22b4c1d9aeb74b0b8e99ffe5e616be63: Bootstrap starting.
I20260809 08:56:31.747160 11154 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22b4c1d9aeb74b0b8e99ffe5e616be63: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:31.747447 11154 log.cc:824] T 00000000000000000000000000000000 P 22b4c1d9aeb74b0b8e99ffe5e616be63: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:31.748317 11154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22b4c1d9aeb74b0b8e99ffe5e616be63: No bootstrap required, opened a new log
I20260809 08:56:31.749974 11159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:31.742495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55921 (local address 127.8.154.62:43639)
0809 08:56:31.742731 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:31.742740 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:31.747856 (+ 5116us) server_negotiation.cc:407] Connection header received
0809 08:56:31.748535 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:31.748545 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:31.748580 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:31.748733 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:31.749528 (+ 795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:31.749532 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:31.749532 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:56:31.749537 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:31.749583 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:56:31.749596 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:31.749695 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:31.749738 (+ 43us) server_negotiation.cc:299] Negotiation successful
0809 08:56:31.749778 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":94,"threads_started":1}
I20260809 08:56:31.751055 8808 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:56:31.751139 11154 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22b4c1d9aeb74b0b8e99ffe5e616be63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22b4c1d9aeb74b0b8e99ffe5e616be63" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 43639 } }
I20260809 08:56:31.751292 11154 raft_consensus.cc:374] T 00000000000000000000000000000000 P 22b4c1d9aeb74b0b8e99ffe5e616be63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:31.751323 11154 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22b4c1d9aeb74b0b8e99ffe5e616be63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22b4c1d9aeb74b0b8e99ffe5e616be63, State: Initialized, Role: FOLLOWER
I20260809 08:56:31.751430 11154 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22b4c1d9aeb74b0b8e99ffe5e616be63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22b4c1d9aeb74b0b8e99ffe5e616be63" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 43639 } }
I20260809 08:56:31.751525 11154 raft_consensus.cc:388] T 00000000000000000000000000000000 P 22b4c1d9aeb74b0b8e99ffe5e616be63 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:56:31.751559 11154 raft_consensus.cc:482] T 00000000000000000000000000000000 P 22b4c1d9aeb74b0b8e99ffe5e616be63 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:56:31.751608 11154 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22b4c1d9aeb74b0b8e99ffe5e616be63 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:31.752337 11154 raft_consensus.cc:504] T 00000000000000000000000000000000 P 22b4c1d9aeb74b0b8e99ffe5e616be63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22b4c1d9aeb74b0b8e99ffe5e616be63" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 43639 } }
I20260809 08:56:31.752462 11154 leader_election.cc:302] T 00000000000000000000000000000000 P 22b4c1d9aeb74b0b8e99ffe5e616be63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 22b4c1d9aeb74b0b8e99ffe5e616be63; no voters:
I20260809 08:56:31.752640 11154 leader_election.cc:288] T 00000000000000000000000000000000 P 22b4c1d9aeb74b0b8e99ffe5e616be63 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:56:31.752660 11168 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 22b4c1d9aeb74b0b8e99ffe5e616be63 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:56:31.752816 11168 raft_consensus.cc:686] T 00000000000000000000000000000000 P 22b4c1d9aeb74b0b8e99ffe5e616be63 [term 1 LEADER]: Becoming Leader. State: Replica: 22b4c1d9aeb74b0b8e99ffe5e616be63, State: Running, Role: LEADER
I20260809 08:56:31.752892 11154 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22b4c1d9aeb74b0b8e99ffe5e616be63 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:56:31.752902 11168 consensus_queue.cc:237] T 00000000000000000000000000000000 P 22b4c1d9aeb74b0b8e99ffe5e616be63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22b4c1d9aeb74b0b8e99ffe5e616be63" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 43639 } }
I20260809 08:56:31.753338 11178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22b4c1d9aeb74b0b8e99ffe5e616be63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "22b4c1d9aeb74b0b8e99ffe5e616be63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22b4c1d9aeb74b0b8e99ffe5e616be63" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 43639 } } }
I20260809 08:56:31.753468 11178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22b4c1d9aeb74b0b8e99ffe5e616be63 [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:31.753328 11169 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22b4c1d9aeb74b0b8e99ffe5e616be63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 22b4c1d9aeb74b0b8e99ffe5e616be63. Latest consensus state: current_term: 1 leader_uuid: "22b4c1d9aeb74b0b8e99ffe5e616be63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "22b4c1d9aeb74b0b8e99ffe5e616be63" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 43639 } } }
I20260809 08:56:31.753826 11169 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22b4c1d9aeb74b0b8e99ffe5e616be63 [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:31.754518 11159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:31.752685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47753 (local address 127.8.154.62:43639)
0809 08:56:31.752723 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:31.752725 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:31.753200 (+ 475us) server_negotiation.cc:407] Connection header received
0809 08:56:31.753499 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:31.753500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:31.753504 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:31.753537 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:31.754230 (+ 693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:31.754231 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:31.754232 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:56:31.754235 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:31.754256 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:56:31.754260 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:31.754330 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:31.754425 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 08:56:31.754440 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 08:56:31.754722 11188 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:56:31.755241 11188 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:56:31.756735 11188 catalog_manager.cc:1347] Generated new cluster ID: 623963c6a11b4a4889fbaff37f8484ec
I20260809 08:56:31.756788 11188 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:56:31.777614 11188 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:56:31.777814 11188 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:56:31.790207 11188 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 22b4c1d9aeb74b0b8e99ffe5e616be63: Generated new TSK 0
I20260809 08:56:31.790488 11188 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:56:31.797159 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 11033
2026-08-09T08:56:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1546 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:56:31 dist-test-slave-t3q3 krb5kdc[11209](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:56:31 dist-test-slave-t3q3 krb5kdc[11209](info): set up 2 sockets
Aug 09 08:56:31 dist-test-slave-t3q3 krb5kdc[11209](info): commencing operation
krb5kdc: starting...
W20260809 08:56:33.041195 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:56:33 dist-test-slave-t3q3 krb5kdc[11209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:56:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:56:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
I20260809 08:56:33.166373 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:38845
--webserver_interface=127.8.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:41729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:38845
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:56:33.248795 11225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:33.248996 11225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:33.249019 11225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:33.250164 11225 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:56:33.250207 11225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:33.250222 11225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:33.250234 11225 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:56:33.250283 11225 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:56:33.251770 11225 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:41729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:38845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:38845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:33.252022 11225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:33.252269 11225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:33.257174 11239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:33.257208 11236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:33.257270 11225 server_base.cc:1034] running on GCE node
W20260809 08:56:33.257190 11237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:33.257656 11225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:33.257932 11225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:33.259071 11225 hybrid_clock.cc:648] HybridClock initialized: now 1786265793259060 us; error 29 us; skew 500 ppm
Aug 09 08:56:33 dist-test-slave-t3q3 krb5kdc[11209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265793, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:56:33.264396 11225 init.cc:377] Logged in from keytab as kudu/127.8.154.62@KRBTEST.COM (short username kudu)
I20260809 08:56:33.264874 11225 webserver.cc:460] Webserver started at http://127.8.154.62:36837/ using document root <none> and password file <none>
I20260809 08:56:33.265147 11225 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:33.265228 11225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:33.265393 11225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:33.266525 11225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "098c1d8e3d034f7caec829aca4e1abfa"
format_stamp: "Formatted at 2026-08-09 08:56:33 on dist-test-slave-t3q3"
server_key: "812ac5d199ba573583f90cbc64b95a18"
server_key_iv: "6f04f1c61818b2429b9c49b09daa8955"
server_key_version: "encryptionkey@0"
I20260809 08:56:33.266943 11225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "098c1d8e3d034f7caec829aca4e1abfa"
format_stamp: "Formatted at 2026-08-09 08:56:33 on dist-test-slave-t3q3"
server_key: "812ac5d199ba573583f90cbc64b95a18"
server_key_iv: "6f04f1c61818b2429b9c49b09daa8955"
server_key_version: "encryptionkey@0"
I20260809 08:56:33.268473 11225 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:56:33.269269 11248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:33.269449 11225 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:56:33.269580 11225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "098c1d8e3d034f7caec829aca4e1abfa"
format_stamp: "Formatted at 2026-08-09 08:56:33 on dist-test-slave-t3q3"
server_key: "812ac5d199ba573583f90cbc64b95a18"
server_key_iv: "6f04f1c61818b2429b9c49b09daa8955"
server_key_version: "encryptionkey@0"
I20260809 08:56:33.269662 11225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:33.325495 11225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:33.327567 11225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:33.327757 11225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:33.332207 11225 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:38845
I20260809 08:56:33.332271 11346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:38845 every 8 connection(s)
I20260809 08:56:33.332686 11225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:56:33.333627 11347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:33.335922 11347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 098c1d8e3d034f7caec829aca4e1abfa: Bootstrap starting.
I20260809 08:56:33.336630 11347 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 098c1d8e3d034f7caec829aca4e1abfa: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:33.336922 11347 log.cc:824] T 00000000000000000000000000000000 P 098c1d8e3d034f7caec829aca4e1abfa: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:33.337678 11347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 098c1d8e3d034f7caec829aca4e1abfa: No bootstrap required, opened a new log
I20260809 08:56:33.339244 11347 raft_consensus.cc:348] T 00000000000000000000000000000000 P 098c1d8e3d034f7caec829aca4e1abfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "098c1d8e3d034f7caec829aca4e1abfa" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 38845 } }
I20260809 08:56:33.339385 11347 raft_consensus.cc:374] T 00000000000000000000000000000000 P 098c1d8e3d034f7caec829aca4e1abfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:33.339406 11347 raft_consensus.cc:729] T 00000000000000000000000000000000 P 098c1d8e3d034f7caec829aca4e1abfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 098c1d8e3d034f7caec829aca4e1abfa, State: Initialized, Role: FOLLOWER
I20260809 08:56:33.339601 11347 consensus_queue.cc:260] T 00000000000000000000000000000000 P 098c1d8e3d034f7caec829aca4e1abfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "098c1d8e3d034f7caec829aca4e1abfa" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 38845 } }
I20260809 08:56:33.339689 11347 raft_consensus.cc:388] T 00000000000000000000000000000000 P 098c1d8e3d034f7caec829aca4e1abfa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:56:33.339732 11347 raft_consensus.cc:482] T 00000000000000000000000000000000 P 098c1d8e3d034f7caec829aca4e1abfa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:56:33.339782 11347 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 098c1d8e3d034f7caec829aca4e1abfa [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:33.340699 11347 raft_consensus.cc:504] T 00000000000000000000000000000000 P 098c1d8e3d034f7caec829aca4e1abfa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "098c1d8e3d034f7caec829aca4e1abfa" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 38845 } }
I20260809 08:56:33.340793 11347 leader_election.cc:302] T 00000000000000000000000000000000 P 098c1d8e3d034f7caec829aca4e1abfa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 098c1d8e3d034f7caec829aca4e1abfa; no voters:
I20260809 08:56:33.340994 11347 leader_election.cc:288] T 00000000000000000000000000000000 P 098c1d8e3d034f7caec829aca4e1abfa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:56:33.341051 11352 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 098c1d8e3d034f7caec829aca4e1abfa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:56:33.341275 11347 sys_catalog.cc:564] T 00000000000000000000000000000000 P 098c1d8e3d034f7caec829aca4e1abfa [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:56:33.341301 11352 raft_consensus.cc:686] T 00000000000000000000000000000000 P 098c1d8e3d034f7caec829aca4e1abfa [term 1 LEADER]: Becoming Leader. State: Replica: 098c1d8e3d034f7caec829aca4e1abfa, State: Running, Role: LEADER
I20260809 08:56:33.341392 11352 consensus_queue.cc:237] T 00000000000000000000000000000000 P 098c1d8e3d034f7caec829aca4e1abfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "098c1d8e3d034f7caec829aca4e1abfa" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 38845 } }
I20260809 08:56:33.341837 11353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 098c1d8e3d034f7caec829aca4e1abfa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "098c1d8e3d034f7caec829aca4e1abfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "098c1d8e3d034f7caec829aca4e1abfa" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 38845 } } }
I20260809 08:56:33.341941 11353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 098c1d8e3d034f7caec829aca4e1abfa [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:33.342135 11355 sys_catalog.cc:455] T 00000000000000000000000000000000 P 098c1d8e3d034f7caec829aca4e1abfa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 098c1d8e3d034f7caec829aca4e1abfa. Latest consensus state: current_term: 1 leader_uuid: "098c1d8e3d034f7caec829aca4e1abfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "098c1d8e3d034f7caec829aca4e1abfa" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 38845 } } }
I20260809 08:56:33.342206 11355 sys_catalog.cc:458] T 00000000000000000000000000000000 P 098c1d8e3d034f7caec829aca4e1abfa [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:33.342185 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 11225
I20260809 08:56:33.342308 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:56:33.342521 8808 external_mini_cluster.cc:1428] Setting key ab00effbb3907d1fa9d326964e937032
I20260809 08:56:33.342551 11366 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:56:33.343215 11366 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:56:33.345049 11366 catalog_manager.cc:1347] Generated new cluster ID: dbb40e1dedfb4128a32b67c6f5abf402
I20260809 08:56:33.345104 11366 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 08:56:33 dist-test-slave-t3q3 krb5kdc[11209](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265793, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:56:33.354946 11383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:33.343242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37827 (local address 127.8.154.62:38845)
0809 08:56:33.343413 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:33.343422 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:33.343457 (+ 35us) server_negotiation.cc:407] Connection header received
0809 08:56:33.343575 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:33.343582 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:33.343638 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:33.343788 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:33.344656 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:33.346128 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:33.347253 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:33.347904 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:33.348026 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:33.350958 (+ 2932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:33.350963 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:33.350963 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:33.350976 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:33.353118 (+ 2142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:33.353515 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:33.353519 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:33.353519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:33.353649 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:33.353860 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:33.353861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:33.353861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:33.354197 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:33.354362 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:33.354433 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 08:56:33.354692 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":77,"threads_started":1}
I20260809 08:56:33.356014 8808 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:56:33.364692 11383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:33.357475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35805 (local address 127.8.154.62:38845)
0809 08:56:33.357509 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:33.357511 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:33.357800 (+ 289us) server_negotiation.cc:407] Connection header received
0809 08:56:33.357829 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:33.357830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:33.357841 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:33.357902 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:33.358490 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:33.359265 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:33.360254 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:33.360591 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:33.360663 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:33.361935 (+ 1272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:33.361937 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:33.361937 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:33.361944 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:33.363538 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:33.363904 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:33.363905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:33.363905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:33.363966 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:33.364189 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:33.364190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:33.364190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:33.364324 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:33.364425 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:33.364525 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 08:56:33.364601 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 08:56:33.380869 11366 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:56:33.381397 11366 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:56:33.387655 11366 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 098c1d8e3d034f7caec829aca4e1abfa: Generated new TSK 0
I20260809 08:56:33.387864 11366 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:56:33.393913 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 11225
2026-08-09T08:56:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1599 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:56:33 dist-test-slave-t3q3 krb5kdc[11397](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:56:33 dist-test-slave-t3q3 krb5kdc[11397](info): set up 2 sockets
Aug 09 08:56:33 dist-test-slave-t3q3 krb5kdc[11397](info): commencing operation
krb5kdc: starting...
W20260809 08:56:34.651111 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.228s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:56:34 dist-test-slave-t3q3 krb5kdc[11397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265794, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:56:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:56:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
I20260809 08:56:34.783406 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:33903
--webserver_interface=127.8.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:43959
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:33903
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:56:34.871603 11414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:34.871860 11414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:34.871901 11414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:34.873147 11414 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:56:34.873224 11414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:34.873251 11414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:34.873281 11414 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:56:34.873308 11414 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:56:34.874788 11414 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:43959
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:33903
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:33903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:34.875082 11414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:34.875319 11414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:34.880455 11428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:34.880517 11432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:34.880517 11427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:34.880548 11414 server_base.cc:1034] running on GCE node
I20260809 08:56:34.881086 11414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:34.881392 11414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:34.882546 11414 hybrid_clock.cc:648] HybridClock initialized: now 1786265794882523 us; error 45 us; skew 500 ppm
Aug 09 08:56:34 dist-test-slave-t3q3 krb5kdc[11397](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265794, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:56:34.885249 11414 init.cc:377] Logged in from keytab as kudu/127.8.154.62@KRBTEST.COM (short username kudu)
I20260809 08:56:34.885731 11414 webserver.cc:460] Webserver started at http://127.8.154.62:40363/ using document root <none> and password file <none>
I20260809 08:56:34.885969 11414 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:34.886039 11414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:34.886197 11414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:34.887254 11414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "47229bbfb835491c8d9a87a79309c364"
format_stamp: "Formatted at 2026-08-09 08:56:34 on dist-test-slave-t3q3"
server_key: "453d1cf74816af39705bf130f8ddbeee"
server_key_iv: "76f4f283db77532bbfbfd84ba24bfd0e"
server_key_version: "encryptionkey@0"
I20260809 08:56:34.887634 11414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "47229bbfb835491c8d9a87a79309c364"
format_stamp: "Formatted at 2026-08-09 08:56:34 on dist-test-slave-t3q3"
server_key: "453d1cf74816af39705bf130f8ddbeee"
server_key_iv: "76f4f283db77532bbfbfd84ba24bfd0e"
server_key_version: "encryptionkey@0"
I20260809 08:56:34.889093 11414 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 08:56:34.889879 11441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:34.890107 11414 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:56:34.890215 11414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "47229bbfb835491c8d9a87a79309c364"
format_stamp: "Formatted at 2026-08-09 08:56:34 on dist-test-slave-t3q3"
server_key: "453d1cf74816af39705bf130f8ddbeee"
server_key_iv: "76f4f283db77532bbfbfd84ba24bfd0e"
server_key_version: "encryptionkey@0"
I20260809 08:56:34.890300 11414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:34.903445 11414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:34.905561 11414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:34.905750 11414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:34.909920 11414 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:33903
I20260809 08:56:34.909960 11516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:33903 every 8 connection(s)
I20260809 08:56:34.910341 11414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:56:34.911293 11517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:34.913779 11517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47229bbfb835491c8d9a87a79309c364: Bootstrap starting.
I20260809 08:56:34.914417 11517 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47229bbfb835491c8d9a87a79309c364: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:34.914719 11517 log.cc:824] T 00000000000000000000000000000000 P 47229bbfb835491c8d9a87a79309c364: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:34.915499 11517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47229bbfb835491c8d9a87a79309c364: No bootstrap required, opened a new log
I20260809 08:56:34.917128 11517 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47229bbfb835491c8d9a87a79309c364 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47229bbfb835491c8d9a87a79309c364" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 33903 } }
I20260809 08:56:34.917281 11517 raft_consensus.cc:374] T 00000000000000000000000000000000 P 47229bbfb835491c8d9a87a79309c364 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:34.917305 11517 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47229bbfb835491c8d9a87a79309c364 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47229bbfb835491c8d9a87a79309c364, State: Initialized, Role: FOLLOWER
I20260809 08:56:34.917502 11517 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47229bbfb835491c8d9a87a79309c364 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47229bbfb835491c8d9a87a79309c364" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 33903 } }
I20260809 08:56:34.917589 11517 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47229bbfb835491c8d9a87a79309c364 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:56:34.917613 11517 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47229bbfb835491c8d9a87a79309c364 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:56:34.917680 11517 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47229bbfb835491c8d9a87a79309c364 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:34.918545 11517 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47229bbfb835491c8d9a87a79309c364 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47229bbfb835491c8d9a87a79309c364" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 33903 } }
I20260809 08:56:34.918637 11517 leader_election.cc:302] T 00000000000000000000000000000000 P 47229bbfb835491c8d9a87a79309c364 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47229bbfb835491c8d9a87a79309c364; no voters:
I20260809 08:56:34.918831 11517 leader_election.cc:288] T 00000000000000000000000000000000 P 47229bbfb835491c8d9a87a79309c364 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:56:34.918891 11521 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47229bbfb835491c8d9a87a79309c364 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:56:34.919107 11517 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47229bbfb835491c8d9a87a79309c364 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:56:34.919147 11521 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47229bbfb835491c8d9a87a79309c364 [term 1 LEADER]: Becoming Leader. State: Replica: 47229bbfb835491c8d9a87a79309c364, State: Running, Role: LEADER
I20260809 08:56:34.919265 11521 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47229bbfb835491c8d9a87a79309c364 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47229bbfb835491c8d9a87a79309c364" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 33903 } }
I20260809 08:56:34.919678 11522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47229bbfb835491c8d9a87a79309c364 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47229bbfb835491c8d9a87a79309c364" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47229bbfb835491c8d9a87a79309c364" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 33903 } } }
I20260809 08:56:34.919790 11522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47229bbfb835491c8d9a87a79309c364 [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:34.919813 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 11414
I20260809 08:56:34.919936 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:56:34.919961 11523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47229bbfb835491c8d9a87a79309c364 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47229bbfb835491c8d9a87a79309c364. Latest consensus state: current_term: 1 leader_uuid: "47229bbfb835491c8d9a87a79309c364" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47229bbfb835491c8d9a87a79309c364" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 33903 } } }
I20260809 08:56:34.920014 11523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47229bbfb835491c8d9a87a79309c364 [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:34.920181 8808 external_mini_cluster.cc:1428] Setting key 6f1736dd623c85135a71db1ad2f794c4
I20260809 08:56:34.920473 11533 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:56:34.921205 11533 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:56:34.922820 11533 catalog_manager.cc:1347] Generated new cluster ID: 6ca99f9101734126b10603ea85f8d982
I20260809 08:56:34.922873 11533 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:56:34.926433 11548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:34.920918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53825 (local address 127.8.154.62:33903)
0809 08:56:34.921114 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:34.921125 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:34.921156 (+ 31us) server_negotiation.cc:407] Connection header received
0809 08:56:34.921281 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:34.921288 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:34.921348 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:34.921495 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:34.921761 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:34.923061 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:34.924215 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:34.924824 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:34.924921 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:34.925645 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:34.925649 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:34.925649 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:56:34.925852 (+ 203us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:34.925895 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:56:34.925910 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:34.925971 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:34.926078 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 08:56:34.926230 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":87,"threads_started":1}
I20260809 08:56:34.927162 8808 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:56:34.932160 11548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:34.928578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46003 (local address 127.8.154.62:33903)
0809 08:56:34.928634 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:34.928636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:34.928844 (+ 208us) server_negotiation.cc:407] Connection header received
0809 08:56:34.928901 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:34.928902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:34.928913 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:34.928966 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:34.929107 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:34.929826 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:34.930836 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:34.931136 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:34.931193 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:34.931880 (+ 687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:34.931882 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:34.931882 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:56:34.931886 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:34.931901 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:56:34.931906 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:34.931955 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:34.932032 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 08:56:34.932088 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 08:56:34.934118 11533 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:56:34.934625 11533 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:56:34.937281 11533 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 47229bbfb835491c8d9a87a79309c364: Generated new TSK 0
I20260809 08:56:34.937462 11533 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:56:34.940201 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 11414
2026-08-09T08:56:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1546 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:56:35 dist-test-slave-t3q3 krb5kdc[11564](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:56:35 dist-test-slave-t3q3 krb5kdc[11564](info): set up 2 sockets
Aug 09 08:56:35 dist-test-slave-t3q3 krb5kdc[11564](info): commencing operation
krb5kdc: starting...
W20260809 08:56:36.250680 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.280s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:56:36 dist-test-slave-t3q3 krb5kdc[11564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265796, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:56:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:56:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
I20260809 08:56:36.372998 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:34867
--webserver_interface=127.8.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:39963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:34867
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:56:36.455632 11588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:36.455838 11588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:36.455862 11588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:36.456965 11588 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:56:36.457001 11588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:36.457015 11588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:36.457028 11588 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:56:36.457041 11588 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:56:36.458429 11588 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:39963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:34867
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:34867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:36.458622 11588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:36.458786 11588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:36.463637 11604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:36.463663 11599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:36.463760 11588 server_base.cc:1034] running on GCE node
W20260809 08:56:36.463639 11600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:36.464212 11588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:36.464517 11588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:36.465690 11588 hybrid_clock.cc:648] HybridClock initialized: now 1786265796465684 us; error 40 us; skew 500 ppm
Aug 09 08:56:36 dist-test-slave-t3q3 krb5kdc[11564](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265796, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:56:36.468230 11588 init.cc:377] Logged in from keytab as kudu/127.8.154.62@KRBTEST.COM (short username kudu)
I20260809 08:56:36.468667 11588 webserver.cc:460] Webserver started at http://127.8.154.62:41977/ using document root <none> and password file <none>
I20260809 08:56:36.468891 11588 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:36.468959 11588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:36.469095 11588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:36.470147 11588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "d1fecd14d06b425c86c06f7dee506031"
format_stamp: "Formatted at 2026-08-09 08:56:36 on dist-test-slave-t3q3"
server_key: "a667d65c5c8351892f61eacb08a8c31f"
server_key_iv: "ec5131c6406986b2b5adb015830c83e2"
server_key_version: "encryptionkey@0"
I20260809 08:56:36.470528 11588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "d1fecd14d06b425c86c06f7dee506031"
format_stamp: "Formatted at 2026-08-09 08:56:36 on dist-test-slave-t3q3"
server_key: "a667d65c5c8351892f61eacb08a8c31f"
server_key_iv: "ec5131c6406986b2b5adb015830c83e2"
server_key_version: "encryptionkey@0"
I20260809 08:56:36.471876 11588 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:56:36.472710 11614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:36.472896 11588 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:56:36.473012 11588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "d1fecd14d06b425c86c06f7dee506031"
format_stamp: "Formatted at 2026-08-09 08:56:36 on dist-test-slave-t3q3"
server_key: "a667d65c5c8351892f61eacb08a8c31f"
server_key_iv: "ec5131c6406986b2b5adb015830c83e2"
server_key_version: "encryptionkey@0"
I20260809 08:56:36.473109 11588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:36.476356 11588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:36.478138 11588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:36.478294 11588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:36.482259 11588 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:34867
I20260809 08:56:36.482290 11688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:34867 every 8 connection(s)
I20260809 08:56:36.482610 11588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:56:36.483583 11689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:36.485827 11689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1fecd14d06b425c86c06f7dee506031: Bootstrap starting.
I20260809 08:56:36.486407 11689 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1fecd14d06b425c86c06f7dee506031: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:36.486654 11689 log.cc:824] T 00000000000000000000000000000000 P d1fecd14d06b425c86c06f7dee506031: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:36.487313 11689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1fecd14d06b425c86c06f7dee506031: No bootstrap required, opened a new log
I20260809 08:56:36.487941 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 11588
I20260809 08:56:36.488094 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:56:36.488270 8808 external_mini_cluster.cc:1428] Setting key 8c4dfc7676a97ba3054bc0e12282e935
I20260809 08:56:36.489475 11689 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1fecd14d06b425c86c06f7dee506031 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1fecd14d06b425c86c06f7dee506031" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 34867 } }
I20260809 08:56:36.489657 11689 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1fecd14d06b425c86c06f7dee506031 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:56:36.489698 11689 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1fecd14d06b425c86c06f7dee506031 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1fecd14d06b425c86c06f7dee506031, State: Initialized, Role: FOLLOWER
I20260809 08:56:36.489886 11689 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1fecd14d06b425c86c06f7dee506031 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1fecd14d06b425c86c06f7dee506031" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 34867 } }
I20260809 08:56:36.489995 11689 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1fecd14d06b425c86c06f7dee506031 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:56:36.490028 11689 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1fecd14d06b425c86c06f7dee506031 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:56:36.490125 11689 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1fecd14d06b425c86c06f7dee506031 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:36.490919 11695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:36.488802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46501 (local address 127.8.154.62:34867)
0809 08:56:36.489027 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:36.489045 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:36.489080 (+ 35us) server_negotiation.cc:407] Connection header received
0809 08:56:36.489432 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:36.489440 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:36.489484 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:36.489636 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:36.490357 (+ 721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:36.490361 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:36.490361 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:56:36.490366 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:36.490412 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:56:36.490428 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:36.490514 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:36.490554 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 08:56:36.490682 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":113,"threads_started":1}
I20260809 08:56:36.491448 11689 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1fecd14d06b425c86c06f7dee506031 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1fecd14d06b425c86c06f7dee506031" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 34867 } }
I20260809 08:56:36.491590 11689 leader_election.cc:302] T 00000000000000000000000000000000 P d1fecd14d06b425c86c06f7dee506031 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1fecd14d06b425c86c06f7dee506031; no voters:
I20260809 08:56:36.491753 11689 leader_election.cc:288] T 00000000000000000000000000000000 P d1fecd14d06b425c86c06f7dee506031 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:56:36.491797 11696 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1fecd14d06b425c86c06f7dee506031 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:56:36.491945 11696 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1fecd14d06b425c86c06f7dee506031 [term 1 LEADER]: Becoming Leader. State: Replica: d1fecd14d06b425c86c06f7dee506031, State: Running, Role: LEADER
I20260809 08:56:36.491945 8808 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:56:36.491992 11689 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1fecd14d06b425c86c06f7dee506031 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:56:36.492038 11696 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1fecd14d06b425c86c06f7dee506031 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1fecd14d06b425c86c06f7dee506031" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 34867 } }
I20260809 08:56:36.492445 11696 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1fecd14d06b425c86c06f7dee506031 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1fecd14d06b425c86c06f7dee506031. Latest consensus state: current_term: 1 leader_uuid: "d1fecd14d06b425c86c06f7dee506031" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1fecd14d06b425c86c06f7dee506031" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 34867 } } }
I20260809 08:56:36.492671 11696 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1fecd14d06b425c86c06f7dee506031 [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:36.492588 11698 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1fecd14d06b425c86c06f7dee506031 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d1fecd14d06b425c86c06f7dee506031" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1fecd14d06b425c86c06f7dee506031" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 34867 } } }
I20260809 08:56:36.493085 11698 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1fecd14d06b425c86c06f7dee506031 [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:36.493273 11708 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:56:36.493872 11708 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:56:36.495263 11695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:36.493588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51313 (local address 127.8.154.62:34867)
0809 08:56:36.493624 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:36.493626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:36.493890 (+ 264us) server_negotiation.cc:407] Connection header received
0809 08:56:36.493992 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:36.493993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:36.493999 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:36.494063 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:36.494942 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:36.494943 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:36.494944 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:56:36.494948 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:36.494963 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:56:36.494967 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:36.495009 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:36.495187 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 08:56:36.495201 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 08:56:36.495612 11708 catalog_manager.cc:1347] Generated new cluster ID: 1103f6ab57474b29a892c7955b2dc0ef
I20260809 08:56:36.495751 11708 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:56:36.519635 11708 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:56:36.519834 11708 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:56:36.522667 11708 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d1fecd14d06b425c86c06f7dee506031: Generated new TSK 0
I20260809 08:56:36.522857 11708 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:56:36.524695 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 11588
2026-08-09T08:56:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1581 ms)
[----------] 18 tests from AuthTokenIssuingTest (28022 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:56:36 dist-test-slave-t3q3 krb5kdc[11739](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:56:36 dist-test-slave-t3q3 krb5kdc[11739](info): set up 2 sockets
Aug 09 08:56:36 dist-test-slave-t3q3 krb5kdc[11739](info): commencing operation
krb5kdc: starting...
W20260809 08:56:37.801106 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.249s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:56:37 dist-test-slave-t3q3 krb5kdc[11739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:56:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:56:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
I20260809 08:56:37.936255 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:33505
--webserver_interface=127.8.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:33315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:33505,127.8.154.61:40123,127.8.154.60:33545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:56:38.019037 11756 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 08:56:38.019249 11756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:38.019268 11756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:38.019289 11756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:38.020507 11756 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:56:38.020552 11756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:38.020567 11756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:38.020581 11756 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:56:38.020593 11756 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:56:38.022075 11756 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:33315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:33505,127.8.154.61:40123,127.8.154.60:33545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:33505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:38.022265 11756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:38.022442 11756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 08:56:38.027726 11756 server_base.cc:1034] running on GCE node
W20260809 08:56:38.027704 11773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:38.027695 11770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:38.027706 11771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:38.028301 11756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:38.028643 11756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:38.029824 11756 hybrid_clock.cc:648] HybridClock initialized: now 1786265798029820 us; error 44 us; skew 500 ppm
Aug 09 08:56:38 dist-test-slave-t3q3 krb5kdc[11739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265798, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:56:38.032564 11756 init.cc:377] Logged in from keytab as kudu/127.8.154.62@KRBTEST.COM (short username kudu)
I20260809 08:56:38.033109 11756 webserver.cc:460] Webserver started at http://127.8.154.62:36255/ using document root <none> and password file <none>
I20260809 08:56:38.033387 11756 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:38.033463 11756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:38.033623 11756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:38.034750 11756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "9038a3d36e6d40b3b609218933b25dbe"
format_stamp: "Formatted at 2026-08-09 08:56:38 on dist-test-slave-t3q3"
server_key: "29367db9425872f54d0b87ec44ee38b1"
server_key_iv: "616a2cfabd104e6142d9c148fcb65971"
server_key_version: "encryptionkey@0"
I20260809 08:56:38.035131 11756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "9038a3d36e6d40b3b609218933b25dbe"
format_stamp: "Formatted at 2026-08-09 08:56:38 on dist-test-slave-t3q3"
server_key: "29367db9425872f54d0b87ec44ee38b1"
server_key_iv: "616a2cfabd104e6142d9c148fcb65971"
server_key_version: "encryptionkey@0"
I20260809 08:56:38.036602 11756 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:56:38.037312 11783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:38.037483 11756 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:56:38.037601 11756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "9038a3d36e6d40b3b609218933b25dbe"
format_stamp: "Formatted at 2026-08-09 08:56:38 on dist-test-slave-t3q3"
server_key: "29367db9425872f54d0b87ec44ee38b1"
server_key_iv: "616a2cfabd104e6142d9c148fcb65971"
server_key_version: "encryptionkey@0"
I20260809 08:56:38.037676 11756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:38.057549 11756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:38.060025 11756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:38.060184 11756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:38.064276 11756 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:33505
I20260809 08:56:38.064332 11877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:33505 every 8 connection(s)
I20260809 08:56:38.064705 11756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:56:38.065786 11884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:38.067402 11884 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.154.62" port: 33505 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:56:38.071224 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 11756
I20260809 08:56:38.071338 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:56:38.071548 8808 external_mini_cluster.cc:1428] Setting key 031c5793687258df6721adc66ec4129b
Aug 09 08:56:38 dist-test-slave-t3q3 krb5kdc[11739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265798, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:56:38.078083 11889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:38.068189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47080 (local address 127.8.154.62:33505)
0809 08:56:38.068376 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:38.068389 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:38.068470 (+ 81us) server_negotiation.cc:407] Connection header received
0809 08:56:38.068974 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:38.068976 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:38.068991 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:38.069073 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:38.069567 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.070409 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.071570 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.071927 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.072009 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:38.074681 (+ 2672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:38.074690 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:38.074690 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:38.074698 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:38.076444 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:38.076929 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:38.076932 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:38.076933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:38.077011 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:38.077282 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:38.077283 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:38.077283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:38.077487 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:38.077550 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:38.077811 (+ 261us) server_negotiation.cc:299] Negotiation successful
0809 08:56:38.077924 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":148,"thread_start_us":122,"threads_started":1}
I20260809 08:56:38.078217 11887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:38.068169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:33505 (local address 127.0.0.1:47080)
0809 08:56:38.068354 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:38.068395 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:38.068846 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:38.069077 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:38.069079 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:38.069325 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:56:38.069486 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:38.069490 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.070529 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.070531 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:38.071476 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:38.071478 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.072045 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.072046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:38.072152 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:38.072674 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:56:38.072685 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:56:38.074582 (+ 1897us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:56:38.076524 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:38.076532 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:38.076535 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:38.076837 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:38.077083 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:38.077084 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:38.077084 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:38.077208 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:38.077560 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:56:38.077568 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:56:38.077663 (+ 95us) client_negotiation.cc:769] Sending connection context
0809 08:56:38.077784 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 08:56:38.078025 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":69,"thread_start_us":98,"threads_started":1}
I20260809 08:56:38.078989 11884 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.154.61" port: 40123 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:56:38.079355 11887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:38.079215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.61:40123 (local address 127.0.0.1:34302)
0809 08:56:38.079267 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:38.079324 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.61:40123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 08:56:38.079514 11785 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.154.61:40123: connect: Connection refused (error 111)
I20260809 08:56:38.079979 11887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:38.079869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.61:40123 (local address 127.0.0.1:34316)
0809 08:56:38.079922 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:38.079955 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.61:40123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 08:56:38.080132 11884 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.154.61:40123: Network error: Client connection negotiation failed: client connection to 127.8.154.61:40123: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.8.154.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.61.keytab.
Entry for principal kudu/127.8.154.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.61.keytab.
I20260809 08:56:38.098045 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.61:40123
--webserver_interface=127.8.154.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.61.keytab
--principal=kudu/127.8.154.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:33315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:33505,127.8.154.61:40123,127.8.154.60:33545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:56:38.129406 11884 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.154.61" port: 40123 } attempt: 1
I20260809 08:56:38.129920 11887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:38.129750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.61:40123 (local address 127.0.0.1:34318)
0809 08:56:38.129825 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:38.129875 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.61:40123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 08:56:38.130357 11887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:38.130248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.61:40123 (local address 127.0.0.1:34324)
0809 08:56:38.130306 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:38.130334 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.61:40123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 08:56:38.130461 11884 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.154.61:40123: Network error: Client connection negotiation failed: client connection to 127.8.154.61:40123: connect: Connection refused (error 111)
W20260809 08:56:38.182515 11897 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 08:56:38.182719 11897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:38.182736 11897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:38.182756 11897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:38.183851 11897 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:56:38.183889 11897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:38.183904 11897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:38.183917 11897 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:56:38.183929 11897 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:56:38.185459 11897 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:33315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:33505,127.8.154.61:40123,127.8.154.60:33545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.61.keytab
--principal=kudu/127.8.154.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.61:40123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.154.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:38.185658 11897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:38.185823 11897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:38.190649 11916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:38.190670 11910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:38.190701 11911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:38.190815 11897 server_base.cc:1034] running on GCE node
I20260809 08:56:38.191123 11897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:38.191447 11897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:38.192606 11897 hybrid_clock.cc:648] HybridClock initialized: now 1786265798192591 us; error 42 us; skew 500 ppm
Aug 09 08:56:38 dist-test-slave-t3q3 krb5kdc[11739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265798, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:56:38.195061 11897 init.cc:377] Logged in from keytab as kudu/127.8.154.61@KRBTEST.COM (short username kudu)
I20260809 08:56:38.195457 11897 webserver.cc:460] Webserver started at http://127.8.154.61:41403/ using document root <none> and password file <none>
I20260809 08:56:38.195657 11897 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:38.195698 11897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:38.195783 11897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:38.196837 11897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-1/data/instance:
uuid: "271cc2660f6d4ef9a4362ac8561403d9"
format_stamp: "Formatted at 2026-08-09 08:56:38 on dist-test-slave-t3q3"
server_key: "21099729bcd74b80759d775e9528fc58"
server_key_iv: "7d8dff88322c6c2a077986908a277521"
server_key_version: "encryptionkey@0"
I20260809 08:56:38.197140 11897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-1/wal/instance:
uuid: "271cc2660f6d4ef9a4362ac8561403d9"
format_stamp: "Formatted at 2026-08-09 08:56:38 on dist-test-slave-t3q3"
server_key: "21099729bcd74b80759d775e9528fc58"
server_key_iv: "7d8dff88322c6c2a077986908a277521"
server_key_version: "encryptionkey@0"
I20260809 08:56:38.198382 11897 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 08:56:38.198583 11884 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.154.61" port: 40123 } attempt: 2
I20260809 08:56:38.199100 11887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:38.198894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.61:40123 (local address 127.0.0.1:34332)
0809 08:56:38.198990 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:38.199048 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.61:40123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 08:56:38.199317 11924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:38.199491 11887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:38.199365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.61:40123 (local address 127.0.0.1:34336)
0809 08:56:38.199416 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:38.199460 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.61:40123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 08:56:38.199519 11897 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
W20260809 08:56:38.199581 11884 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.154.61:40123: Network error: Client connection negotiation failed: client connection to 127.8.154.61:40123: connect: Connection refused (error 111)
I20260809 08:56:38.199620 11897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-1/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-1/wal
uuid: "271cc2660f6d4ef9a4362ac8561403d9"
format_stamp: "Formatted at 2026-08-09 08:56:38 on dist-test-slave-t3q3"
server_key: "21099729bcd74b80759d775e9528fc58"
server_key_iv: "7d8dff88322c6c2a077986908a277521"
server_key_version: "encryptionkey@0"
I20260809 08:56:38.199693 11897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:38.214743 11897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:38.217151 11897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:38.217325 11897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:38.221496 11897 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.61:40123
I20260809 08:56:38.221582 12004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.61:40123 every 8 connection(s)
I20260809 08:56:38.221931 11897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-1/data/info.pb
I20260809 08:56:38.222903 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 11897
I20260809 08:56:38.223042 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-1/wal/instance
I20260809 08:56:38.223315 8808 external_mini_cluster.cc:1428] Setting key 0b23bd0396fd61aa5fb75d74bf02d672
I20260809 08:56:38.223307 12005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:38.225879 12005 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.154.62" port: 33505 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.8.154.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.60@KRBTEST.COM" created.
Aug 09 08:56:38 dist-test-slave-t3q3 krb5kdc[11739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265798, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.61@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:56:38.238915 11889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:38.226684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47084 (local address 127.8.154.62:33505)
0809 08:56:38.226762 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:38.226765 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:38.226968 (+ 203us) server_negotiation.cc:407] Connection header received
0809 08:56:38.227560 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:38.227561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:38.227580 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:38.227653 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:38.228424 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.228994 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.230117 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.230358 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.230442 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:38.235587 (+ 5145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:38.235588 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:38.235589 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:38.235597 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:38.237380 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:38.237825 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:38.237826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:38.237827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:38.237916 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:38.238232 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:38.238233 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:38.238233 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:38.238362 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:38.238433 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:38.238726 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 08:56:38.238816 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 08:56:38.239125 12012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:38.226644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:33505 (local address 127.0.0.1:47084)
0809 08:56:38.226837 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:38.226876 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:38.227434 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:38.227737 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:38.227740 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:38.228029 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:56:38.228316 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:38.228322 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.229103 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.229104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:38.230010 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:38.230013 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.230480 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.230481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:38.230622 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:38.231219 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:56:38.231228 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:56:38.235469 (+ 4241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:56:38.237487 (+ 2018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:38.237495 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:38.237496 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:38.237738 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:38.238013 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:38.238014 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:38.238015 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:38.238133 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:38.238437 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:56:38.238445 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:56:38.238591 (+ 146us) client_negotiation.cc:769] Sending connection context
0809 08:56:38.238703 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 08:56:38.238870 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":86,"threads_started":1}
I20260809 08:56:38.239712 12005 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.154.61" port: 40123 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 08:56:38 dist-test-slave-t3q3 krb5kdc[11739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265798, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.61@KRBTEST.COM for kudu/127.8.154.61@KRBTEST.COM
I20260809 08:56:38.248521 12012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:38.239976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.61:40123 (local address 127.0.0.1:34342)
0809 08:56:38.240024 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:38.240235 (+ 211us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:38.240282 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:38.240463 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:38.240464 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:38.240640 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:56:38.240704 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:38.240706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.241656 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.241657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:38.242151 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:38.242153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.242661 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.242662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:38.242730 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:38.243168 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:56:38.243171 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:56:38.245749 (+ 2578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:56:38.247476 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:38.247477 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:38.247478 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:38.247646 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:38.247849 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:38.247850 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:38.247850 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:38.247903 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:38.248270 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:56:38.248271 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:56:38.248318 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 08:56:38.248376 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 08:56:38.248457 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 08:56:38.248689 12016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:38.240086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34342 (local address 127.8.154.61:40123)
0809 08:56:38.240259 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:38.240264 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:38.240346 (+ 82us) server_negotiation.cc:407] Connection header received
0809 08:56:38.240369 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:38.240371 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:38.240384 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:38.240457 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:38.240785 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.241555 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.242245 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.242564 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.242655 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:38.245861 (+ 3206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:38.245869 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:38.245869 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:38.245875 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:38.247406 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:38.247723 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:38.247725 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:38.247725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:38.247795 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:38.247957 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:38.247957 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:38.247957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:38.248202 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:38.248293 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:38.248400 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 08:56:38.248517 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":141,"thread_start_us":106,"threads_started":1}
I20260809 08:56:38.249374 12005 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.154.60" port: 33545 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:56:38.249739 12012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:38.249628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:33545 (local address 127.0.0.1:51842)
0809 08:56:38.249679 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:38.249722 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.60:33545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 08:56:38.249928 11925 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.154.60:33545: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.60.keytab.
Entry for principal kudu/127.8.154.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.60.keytab.
I20260809 08:56:38.250373 12012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:38.250308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:33545 (local address 127.0.0.1:51858)
0809 08:56:38.250334 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:38.250358 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.60:33545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260809 08:56:38.250494 12005 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.154.60:33545: Network error: Client connection negotiation failed: client connection to 127.8.154.60:33545: connect: Connection refused (error 111)
I20260809 08:56:38.250875 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.60:33545
--webserver_interface=127.8.154.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.60.keytab
--principal=kudu/127.8.154.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:33315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:33505,127.8.154.61:40123,127.8.154.60:33545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:56:38.290745 11884 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.154.61" port: 40123 } attempt: 3
Aug 09 08:56:38 dist-test-slave-t3q3 krb5kdc[11739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265798, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for kudu/127.8.154.61@KRBTEST.COM
I20260809 08:56:38.299728 12005 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.154.60" port: 33545 } attempt: 1
I20260809 08:56:38.300161 12012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:38.299992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:33545 (local address 127.0.0.1:51866)
0809 08:56:38.300047 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:38.300114 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.60:33545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 08:56:38.300549 12012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:38.300448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:33545 (local address 127.0.0.1:51874)
0809 08:56:38.300496 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:38.300522 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.60:33545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 08:56:38.300653 12005 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.154.60:33545: Network error: Client connection negotiation failed: client connection to 127.8.154.60:33545: connect: Connection refused (error 111)
I20260809 08:56:38.301880 11887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:38.291200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.61:40123 (local address 127.0.0.1:34350)
0809 08:56:38.291289 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:38.291322 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:38.291385 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:38.291583 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:38.291584 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:38.291773 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:56:38.291845 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:38.291847 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.292695 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.292696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:38.293296 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:38.293298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.293752 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.293753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:38.293843 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:38.297697 (+ 3854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:56:38.297702 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:56:38.299032 (+ 1330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:56:38.300865 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:38.300866 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:38.300866 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:38.301033 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:38.301236 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:38.301237 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:38.301237 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:38.301291 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:38.301564 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:56:38.301565 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:56:38.301623 (+ 58us) client_negotiation.cc:769] Sending connection context
0809 08:56:38.301679 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 08:56:38.301815 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 08:56:38.301880 12016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:38.291281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34350 (local address 127.8.154.61:40123)
0809 08:56:38.291358 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:38.291361 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:38.291374 (+ 13us) server_negotiation.cc:407] Connection header received
0809 08:56:38.291466 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:38.291467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:38.291497 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:38.291574 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:38.291948 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.292593 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.293400 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.293663 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.293755 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:38.299121 (+ 5366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:38.299123 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:38.299123 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:38.299131 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:38.300768 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:38.301098 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:38.301099 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:38.301100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:38.301170 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:38.301348 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:38.301349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:38.301349 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:38.301485 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:38.301551 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:38.301721 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 08:56:38.301801 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 08:56:38.302277 11884 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.154.60" port: 33545 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:56:38.302585 11887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:38.302490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:33545 (local address 127.0.0.1:51880)
0809 08:56:38.302533 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:38.302563 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.60:33545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260809 08:56:38.302927 11887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:38.302833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:33545 (local address 127.0.0.1:51892)
0809 08:56:38.302881 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:38.302908 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.60:33545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 08:56:38.303045 11884 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.154.60:33545: Network error: Client connection negotiation failed: client connection to 127.8.154.60:33545: connect: Connection refused (error 111)
I20260809 08:56:38.334230 11884 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.154.60" port: 33545 } attempt: 1
I20260809 08:56:38.334685 11887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:38.334552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:33545 (local address 127.0.0.1:51904)
0809 08:56:38.334616 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:38.334656 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.60:33545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 08:56:38.335130 11887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:38.335017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:33545 (local address 127.0.0.1:51906)
0809 08:56:38.335067 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:38.335111 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.60:33545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 08:56:38.335261 11884 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.154.60:33545: Network error: Client connection negotiation failed: client connection to 127.8.154.60:33545: connect: Connection refused (error 111)
W20260809 08:56:38.335667 12018 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 08:56:38.335843 12018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:38.335861 12018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:38.335880 12018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:38.337015 12018 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:56:38.337056 12018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:38.337071 12018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:38.337083 12018 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:56:38.337095 12018 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:56:38.338524 12018 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:33315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:33505,127.8.154.61:40123,127.8.154.60:33545
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.60.keytab
--principal=kudu/127.8.154.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.60:33545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.154.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:38.338721 12018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:38.338891 12018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:38.343688 12024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:38.343779 12027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:38.343688 12025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:38.343981 12018 server_base.cc:1034] running on GCE node
I20260809 08:56:38.344312 12018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:38.344625 12018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:38.345793 12018 hybrid_clock.cc:648] HybridClock initialized: now 1786265798345787 us; error 34 us; skew 500 ppm
Aug 09 08:56:38 dist-test-slave-t3q3 krb5kdc[11739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265798, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:56:38.348315 12018 init.cc:377] Logged in from keytab as kudu/127.8.154.60@KRBTEST.COM (short username kudu)
I20260809 08:56:38.348721 12018 webserver.cc:460] Webserver started at http://127.8.154.60:36793/ using document root <none> and password file <none>
I20260809 08:56:38.348923 12018 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:38.348964 12018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:38.349054 12018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:38.350051 12018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-2/data/instance:
uuid: "e4f23887a015480c8e9f0f7467d2b07c"
format_stamp: "Formatted at 2026-08-09 08:56:38 on dist-test-slave-t3q3"
server_key: "e0c4b5d305db21bdd35d8fa4d66867d7"
server_key_iv: "d4cc082061d55e4f5aa32667d6502161"
server_key_version: "encryptionkey@0"
I20260809 08:56:38.350358 12018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-2/wal/instance:
uuid: "e4f23887a015480c8e9f0f7467d2b07c"
format_stamp: "Formatted at 2026-08-09 08:56:38 on dist-test-slave-t3q3"
server_key: "e0c4b5d305db21bdd35d8fa4d66867d7"
server_key_iv: "d4cc082061d55e4f5aa32667d6502161"
server_key_version: "encryptionkey@0"
I20260809 08:56:38.351718 12018 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:56:38.352496 12037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:38.352706 12018 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 08:56:38.352782 12018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-2/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-2/wal
uuid: "e4f23887a015480c8e9f0f7467d2b07c"
format_stamp: "Formatted at 2026-08-09 08:56:38 on dist-test-slave-t3q3"
server_key: "e0c4b5d305db21bdd35d8fa4d66867d7"
server_key_iv: "d4cc082061d55e4f5aa32667d6502161"
server_key_version: "encryptionkey@0"
I20260809 08:56:38.352831 12018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:38.361752 12018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:38.363591 12018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:38.363713 12018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:38.368216 12018 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.60:33545
I20260809 08:56:38.368281 12126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.60:33545 every 8 connection(s)
I20260809 08:56:38.368567 12018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-2/data/info.pb
I20260809 08:56:38.368811 12005 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.154.60" port: 33545 } attempt: 2
I20260809 08:56:38.369750 12127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:38.371810 12127 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.154.62" port: 33505 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 08:56:38 dist-test-slave-t3q3 krb5kdc[11739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265798, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.61@KRBTEST.COM for kudu/127.8.154.60@KRBTEST.COM
I20260809 08:56:38.376127 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 12018
I20260809 08:56:38.376248 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265519942966-8808-0/minicluster-data/master-2/wal/instance
I20260809 08:56:38.376489 8808 external_mini_cluster.cc:1428] Setting key caee9ff92ff10b97f977a58efc424dfd
Aug 09 08:56:38 dist-test-slave-t3q3 krb5kdc[11739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265798, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.60@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:56:38.379204 12012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:38.369119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:33545 (local address 127.0.0.1:51912)
0809 08:56:38.369190 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:38.369206 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:38.369260 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:38.370476 (+ 1216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:38.370478 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:38.370626 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:56:38.370674 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:38.370676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.372333 (+ 1657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.372335 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:38.373113 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:38.373115 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.373750 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.373751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:38.373811 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:38.374275 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:56:38.374279 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:56:38.375509 (+ 1230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:56:38.377634 (+ 2125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:38.377635 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:38.377635 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:38.377819 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:38.378117 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:38.378117 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:38.378118 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:38.378188 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:38.378659 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:56:38.378660 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:56:38.378710 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 08:56:38.378787 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 08:56:38.379135 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 08:56:38.379294 12130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:38.369344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51912 (local address 127.8.154.60:33545)
0809 08:56:38.369553 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:38.369562 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:38.369593 (+ 31us) server_negotiation.cc:407] Connection header received
0809 08:56:38.370233 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:38.370243 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:38.370306 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:38.370465 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:38.370786 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.372226 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.373203 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.373647 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.373794 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:38.375610 (+ 1816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:38.375619 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:38.375619 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:38.375632 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:38.377531 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:38.377942 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:38.377945 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:38.377945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:38.378041 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:38.378251 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:38.378252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:38.378252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:38.378573 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:38.378668 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:38.378892 (+ 224us) server_negotiation.cc:299] Negotiation successful
0809 08:56:38.379029 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":100,"threads_started":1}
I20260809 08:56:38.381390 11889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:38.372575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47094 (local address 127.8.154.62:33505)
0809 08:56:38.372620 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:38.372622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:38.372823 (+ 201us) server_negotiation.cc:407] Connection header received
0809 08:56:38.372854 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:38.372855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:38.372867 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:38.372917 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:38.373350 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.373818 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.374746 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.374957 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.375045 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:38.377939 (+ 2894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:38.377940 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:38.377940 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:38.377948 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:38.379850 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:38.380401 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:38.380402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:38.380403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:38.380497 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:38.380739 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:38.380740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:38.380740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:38.380878 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:38.380946 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:38.381205 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 08:56:38.381289 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 08:56:38.381407 12131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:38.372512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:33505 (local address 127.0.0.1:47094)
0809 08:56:38.372694 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:38.372726 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:38.372783 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:38.372929 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:38.372931 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:38.373190 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:56:38.373255 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:38.373257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.373903 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.373904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:38.374629 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:38.374632 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.375081 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.375083 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:38.375158 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:38.375717 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:56:38.375730 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:56:38.377833 (+ 2103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:56:38.379973 (+ 2140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:38.379982 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:38.379984 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:38.380308 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:38.380594 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:38.380595 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:38.380595 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:38.380668 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:38.380980 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:56:38.380989 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:56:38.381045 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 08:56:38.381157 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 08:56:38.381289 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":91,"threads_started":1}
I20260809 08:56:38.381753 12005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 271cc2660f6d4ef9a4362ac8561403d9: Bootstrap starting.
I20260809 08:56:38.382107 12127 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.154.61" port: 40123 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:56:38.382853 12005 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 271cc2660f6d4ef9a4362ac8561403d9: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:38.383195 12005 log.cc:824] T 00000000000000000000000000000000 P 271cc2660f6d4ef9a4362ac8561403d9: Log is configured to *not* fsync() on all Append() calls
Aug 09 08:56:38 dist-test-slave-t3q3 krb5kdc[11739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.154.60@KRBTEST.COM
I20260809 08:56:38.384310 12005 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 271cc2660f6d4ef9a4362ac8561403d9: No bootstrap required, opened a new log
I20260809 08:56:38.386353 12005 raft_consensus.cc:348] T 00000000000000000000000000000000 P 271cc2660f6d4ef9a4362ac8561403d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9038a3d36e6d40b3b609218933b25dbe" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 33505 } } peers { permanent_uuid: "271cc2660f6d4ef9a4362ac8561403d9" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 40123 } } peers { permanent_uuid: "e4f23887a015480c8e9f0f7467d2b07c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 33545 } }
I20260809 08:56:38.386534 12005 raft_consensus.cc:729] T 00000000000000000000000000000000 P 271cc2660f6d4ef9a4362ac8561403d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 271cc2660f6d4ef9a4362ac8561403d9, State: Initialized, Role: FOLLOWER
I20260809 08:56:38.386624 12005 consensus_queue.cc:260] T 00000000000000000000000000000000 P 271cc2660f6d4ef9a4362ac8561403d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9038a3d36e6d40b3b609218933b25dbe" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 33505 } } peers { permanent_uuid: "271cc2660f6d4ef9a4362ac8561403d9" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 40123 } } peers { permanent_uuid: "e4f23887a015480c8e9f0f7467d2b07c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 33545 } }
I20260809 08:56:38.386937 12135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 271cc2660f6d4ef9a4362ac8561403d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9038a3d36e6d40b3b609218933b25dbe" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 33505 } } peers { permanent_uuid: "271cc2660f6d4ef9a4362ac8561403d9" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 40123 } } peers { permanent_uuid: "e4f23887a015480c8e9f0f7467d2b07c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 33545 } } }
I20260809 08:56:38.387002 12005 sys_catalog.cc:564] T 00000000000000000000000000000000 P 271cc2660f6d4ef9a4362ac8561403d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:56:38.387151 12135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 271cc2660f6d4ef9a4362ac8561403d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:56:38.387887 12132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:38.376870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51918 (local address 127.8.154.60:33545)
0809 08:56:38.376986 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:38.376988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:38.377140 (+ 152us) server_negotiation.cc:407] Connection header received
0809 08:56:38.377262 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:38.377263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:38.377273 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:38.377315 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:38.378171 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.378890 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.380994 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.381278 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.381349 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:38.384510 (+ 3161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:38.384511 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:38.384511 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:38.384518 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:38.386326 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:38.386744 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:38.386745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:38.386746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:38.386815 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:38.387270 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:38.387271 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:38.387271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:38.387409 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:38.387574 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:38.387653 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 08:56:38.387739 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":68,"threads_started":1}
Aug 09 08:56:38 dist-test-slave-t3q3 krb5kdc[11739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265798, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.60@KRBTEST.COM for kudu/127.8.154.61@KRBTEST.COM
W20260809 08:56:38.389264 12153 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 271cc2660f6d4ef9a4362ac8561403d9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:56:38.389328 12153 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:56:38.391487 12016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:38.382461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34364 (local address 127.8.154.61:40123)
0809 08:56:38.382505 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:38.382508 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:38.382517 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:56:38.382581 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:38.382582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:38.382592 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:38.382654 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:38.382949 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.383782 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.385292 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.385910 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.385991 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:38.388506 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:38.388507 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:38.388508 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:38.388515 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:38.390176 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:38.390579 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:38.390580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:38.390580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:38.390646 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:38.390875 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:38.390876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:38.390876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:38.391055 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:38.391106 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:38.391326 (+ 220us) server_negotiation.cc:299] Negotiation successful
0809 08:56:38.391405 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 08:56:38.391799 12131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:38.382407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.61:40123 (local address 127.0.0.1:34364)
0809 08:56:38.382452 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:38.382462 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:38.382515 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:38.382658 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:38.382659 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:38.382811 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:56:38.382862 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:38.382863 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.383877 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.383879 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:38.385190 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:38.385193 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.386008 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.386010 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:38.386095 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:38.386794 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:56:38.386800 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:56:38.388388 (+ 1588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:56:38.390275 (+ 1887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:38.390276 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:38.390276 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:38.390481 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:38.390724 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:38.390725 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:38.390726 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:38.390792 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:38.391166 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:56:38.391167 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:56:38.391214 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 08:56:38.391612 (+ 398us) client_negotiation.cc:240] Negotiation successful
0809 08:56:38.391714 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 08:56:38.392292 12127 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.154.60" port: 33545 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:56:38.389523 11897 master_runner.cc:186] Error getting master registration for 127.8.154.62:33505: OK, instance_id { permanent_uuid: "9038a3d36e6d40b3b609218933b25dbe" instance_seqno: 1786265798063100 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 08:56:38.392882 11897 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 09 08:56:38 dist-test-slave-t3q3 krb5kdc[11739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:56:38.397472 11889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:38.388531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47100 (local address 127.8.154.62:33505)
0809 08:56:38.388572 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:38.388574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:38.388582 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:56:38.388606 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:38.388607 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:38.388618 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:38.388672 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:38.389834 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.390528 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.391558 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.391867 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.391970 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:38.394649 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:38.394651 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:38.394652 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:38.394659 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:38.396245 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:38.396648 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:38.396650 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:38.396650 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:38.396720 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:38.396942 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:38.396943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:38.396944 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:38.397069 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:38.397257 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:38.397294 (+ 37us) server_negotiation.cc:299] Negotiation successful
0809 08:56:38.397367 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Aug 09 08:56:38 dist-test-slave-t3q3 krb5kdc[11739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265798, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.60@KRBTEST.COM for kudu/127.8.154.60@KRBTEST.COM
I20260809 08:56:38.401327 12131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:38.392543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:33545 (local address 127.0.0.1:51928)
0809 08:56:38.392615 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:38.392624 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:38.392659 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:38.393739 (+ 1080us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:38.393740 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:38.393890 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:56:38.393934 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:38.393937 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.394875 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.394876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:38.395655 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:38.395658 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.396215 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.396215 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:38.396266 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:38.396825 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:56:38.396830 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:56:38.398271 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:56:38.400102 (+ 1831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:38.400103 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:38.400104 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:38.400329 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:38.400556 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:38.400557 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:38.400557 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:38.400626 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:38.400921 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:56:38.400922 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:56:38.400971 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 08:56:38.401063 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 08:56:38.401191 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 08:56:38.401901 12132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:38.393510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51928 (local address 127.8.154.60:33545)
0809 08:56:38.393605 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:38.393607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:38.393617 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:56:38.393657 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:38.393658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:38.393670 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:38.393731 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:38.394042 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.394780 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.395751 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.396096 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.396140 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:38.398361 (+ 2221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:38.398362 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:38.398362 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:38.398369 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:38.399985 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:38.400410 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:38.400411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:38.400411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:38.400473 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:38.400699 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:38.400700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:38.400700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:38.400841 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:38.400912 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:38.401093 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 08:56:38.401775 (+ 682us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":584,"server-negotiator.queue_time_us":72}
Aug 09 08:56:38 dist-test-slave-t3q3 krb5kdc[11739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265797, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.154.61@KRBTEST.COM
I20260809 08:56:38.403834 12127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4f23887a015480c8e9f0f7467d2b07c: Bootstrap starting.
I20260809 08:56:38.404695 12127 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4f23887a015480c8e9f0f7467d2b07c: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:38.405207 12127 log.cc:824] T 00000000000000000000000000000000 P e4f23887a015480c8e9f0f7467d2b07c: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:38.406172 12127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4f23887a015480c8e9f0f7467d2b07c: No bootstrap required, opened a new log
I20260809 08:56:38.406534 12016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:38.398219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34380 (local address 127.8.154.61:40123)
0809 08:56:38.398257 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:38.398258 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:38.398265 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:56:38.398285 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:38.398286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:38.398294 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:38.398363 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:38.399031 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.399742 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.400604 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.400890 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.400951 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:38.403589 (+ 2638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:38.403591 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:38.403591 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:38.403598 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:38.405181 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:38.405645 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:38.405647 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:38.405647 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:38.405713 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:38.405958 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:38.405960 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:38.405961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:38.406112 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:38.406295 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:38.406353 (+ 58us) server_negotiation.cc:299] Negotiation successful
0809 08:56:38.406434 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:56:38.408303 8808 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:56:38.408470 12127 raft_consensus.cc:348] T 00000000000000000000000000000000 P e4f23887a015480c8e9f0f7467d2b07c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9038a3d36e6d40b3b609218933b25dbe" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 33505 } } peers { permanent_uuid: "271cc2660f6d4ef9a4362ac8561403d9" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 40123 } } peers { permanent_uuid: "e4f23887a015480c8e9f0f7467d2b07c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 33545 } }
I20260809 08:56:38.408625 12127 raft_consensus.cc:729] T 00000000000000000000000000000000 P e4f23887a015480c8e9f0f7467d2b07c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4f23887a015480c8e9f0f7467d2b07c, State: Initialized, Role: FOLLOWER
I20260809 08:56:38.408706 12127 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4f23887a015480c8e9f0f7467d2b07c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9038a3d36e6d40b3b609218933b25dbe" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 33505 } } peers { permanent_uuid: "271cc2660f6d4ef9a4362ac8561403d9" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 40123 } } peers { permanent_uuid: "e4f23887a015480c8e9f0f7467d2b07c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 33545 } }
I20260809 08:56:38.409088 12127 sys_catalog.cc:564] T 00000000000000000000000000000000 P e4f23887a015480c8e9f0f7467d2b07c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:56:38.409050 12160 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4f23887a015480c8e9f0f7467d2b07c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9038a3d36e6d40b3b609218933b25dbe" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 33505 } } peers { permanent_uuid: "271cc2660f6d4ef9a4362ac8561403d9" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 40123 } } peers { permanent_uuid: "e4f23887a015480c8e9f0f7467d2b07c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 33545 } } }
I20260809 08:56:38.409161 12160 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4f23887a015480c8e9f0f7467d2b07c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:56:38.410408 11884 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.154.60" port: 33545 } attempt: 2
I20260809 08:56:38.409667 11838 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "9038a3d36e6d40b3b609218933b25dbe"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47100
W20260809 08:56:38.412860 12180 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e4f23887a015480c8e9f0f7467d2b07c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:56:38.412926 12180 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 09 08:56:38 dist-test-slave-t3q3 krb5kdc[11739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265798, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for kudu/127.8.154.60@KRBTEST.COM
I20260809 08:56:38.412271 12018 master_runner.cc:186] Error getting master registration for 127.8.154.62:33505: OK, instance_id { permanent_uuid: "9038a3d36e6d40b3b609218933b25dbe" instance_seqno: 1786265798063100 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 08:56:38.415647 12018 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 08:56:38.418653 12132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:38.410919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51942 (local address 127.8.154.60:33545)
0809 08:56:38.410955 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:38.410958 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:38.410966 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:56:38.410988 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:38.410989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:38.411006 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:38.411052 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:38.411391 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.411878 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.412899 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.413236 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.413306 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:38.415650 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:38.415651 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:38.415651 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:38.415659 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:38.417484 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:38.417853 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:38.417854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:38.417855 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:38.417912 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:38.418126 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:38.418127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:38.418127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:38.418248 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:38.418325 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:38.418482 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 08:56:38.418566 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 08:56:38.418653 11887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:38.410720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:33545 (local address 127.0.0.1:51942)
0809 08:56:38.410777 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:38.410790 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:38.410838 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:38.411082 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:38.411083 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:38.411276 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:56:38.411329 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:38.411331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.411952 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.411953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:38.412795 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:38.412798 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.413322 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.413324 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:38.413393 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:38.413871 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:56:38.413877 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:56:38.415546 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:56:38.417574 (+ 2028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:38.417575 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:38.417576 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:38.417759 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:38.417975 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:38.417990 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:38.417991 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:38.418061 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:38.418328 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:56:38.418330 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:56:38.418379 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 08:56:38.418445 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 08:56:38.418554 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 08:56:38.420013 11884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9038a3d36e6d40b3b609218933b25dbe: Bootstrap starting.
I20260809 08:56:38.420951 11884 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9038a3d36e6d40b3b609218933b25dbe: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:38.421280 11884 log.cc:824] T 00000000000000000000000000000000 P 9038a3d36e6d40b3b609218933b25dbe: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:38.422088 11884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9038a3d36e6d40b3b609218933b25dbe: No bootstrap required, opened a new log
I20260809 08:56:38.423120 11884 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9038a3d36e6d40b3b609218933b25dbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9038a3d36e6d40b3b609218933b25dbe" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 33505 } } peers { permanent_uuid: "271cc2660f6d4ef9a4362ac8561403d9" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 40123 } } peers { permanent_uuid: "e4f23887a015480c8e9f0f7467d2b07c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 33545 } }
I20260809 08:56:38.423266 11884 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9038a3d36e6d40b3b609218933b25dbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9038a3d36e6d40b3b609218933b25dbe, State: Initialized, Role: FOLLOWER
I20260809 08:56:38.423355 11884 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9038a3d36e6d40b3b609218933b25dbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9038a3d36e6d40b3b609218933b25dbe" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 33505 } } peers { permanent_uuid: "271cc2660f6d4ef9a4362ac8561403d9" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 40123 } } peers { permanent_uuid: "e4f23887a015480c8e9f0f7467d2b07c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 33545 } }
I20260809 08:56:38.423652 12183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9038a3d36e6d40b3b609218933b25dbe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9038a3d36e6d40b3b609218933b25dbe" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 33505 } } peers { permanent_uuid: "271cc2660f6d4ef9a4362ac8561403d9" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 40123 } } peers { permanent_uuid: "e4f23887a015480c8e9f0f7467d2b07c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 33545 } } }
I20260809 08:56:38.423760 11838 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9038a3d36e6d40b3b609218933b25dbe [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 08:56:38.423710 11884 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9038a3d36e6d40b3b609218933b25dbe [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:56:38.423818 11838 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9038a3d36e6d40b3b609218933b25dbe [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:38.423787 12183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9038a3d36e6d40b3b609218933b25dbe [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:56:38.424607 11838 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9038a3d36e6d40b3b609218933b25dbe [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9038a3d36e6d40b3b609218933b25dbe" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 33505 } } peers { permanent_uuid: "271cc2660f6d4ef9a4362ac8561403d9" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 40123 } } peers { permanent_uuid: "e4f23887a015480c8e9f0f7467d2b07c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 33545 } }
I20260809 08:56:38.424876 11838 leader_election.cc:288] T 00000000000000000000000000000000 P 9038a3d36e6d40b3b609218933b25dbe [CANDIDATE]: Term 1 election: Requested vote from peers 271cc2660f6d4ef9a4362ac8561403d9 (127.8.154.61:40123), e4f23887a015480c8e9f0f7467d2b07c (127.8.154.60:33545)
I20260809 08:56:38.425227 11967 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9038a3d36e6d40b3b609218933b25dbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "271cc2660f6d4ef9a4362ac8561403d9"
I20260809 08:56:38.425387 11967 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 271cc2660f6d4ef9a4362ac8561403d9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:38.426236 12089 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9038a3d36e6d40b3b609218933b25dbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e4f23887a015480c8e9f0f7467d2b07c"
I20260809 08:56:38.426389 12089 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e4f23887a015480c8e9f0f7467d2b07c [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:38.426404 11967 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 271cc2660f6d4ef9a4362ac8561403d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9038a3d36e6d40b3b609218933b25dbe in term 1.
I20260809 08:56:38.427124 12089 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e4f23887a015480c8e9f0f7467d2b07c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9038a3d36e6d40b3b609218933b25dbe in term 1.
I20260809 08:56:38.427397 11785 leader_election.cc:302] T 00000000000000000000000000000000 P 9038a3d36e6d40b3b609218933b25dbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 271cc2660f6d4ef9a4362ac8561403d9, 9038a3d36e6d40b3b609218933b25dbe; no voters:
I20260809 08:56:38.427629 12183 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9038a3d36e6d40b3b609218933b25dbe [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:56:38.427845 12183 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9038a3d36e6d40b3b609218933b25dbe [term 1 LEADER]: Becoming Leader. State: Replica: 9038a3d36e6d40b3b609218933b25dbe, State: Running, Role: LEADER
I20260809 08:56:38.428004 12183 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9038a3d36e6d40b3b609218933b25dbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9038a3d36e6d40b3b609218933b25dbe" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 33505 } } peers { permanent_uuid: "271cc2660f6d4ef9a4362ac8561403d9" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 40123 } } peers { permanent_uuid: "e4f23887a015480c8e9f0f7467d2b07c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 33545 } }
I20260809 08:56:38.428581 12183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9038a3d36e6d40b3b609218933b25dbe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9038a3d36e6d40b3b609218933b25dbe. Latest consensus state: current_term: 1 leader_uuid: "9038a3d36e6d40b3b609218933b25dbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9038a3d36e6d40b3b609218933b25dbe" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 33505 } } peers { permanent_uuid: "271cc2660f6d4ef9a4362ac8561403d9" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 40123 } } peers { permanent_uuid: "e4f23887a015480c8e9f0f7467d2b07c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 33545 } } }
I20260809 08:56:38.428764 12183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9038a3d36e6d40b3b609218933b25dbe [sys.catalog]: This master's current role is: LEADER
W20260809 08:56:38.429927 12209 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9038a3d36e6d40b3b609218933b25dbe: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:56:38.429982 12209 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:56:38.439881 11889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:38.430506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47114 (local address 127.8.154.62:33505)
0809 08:56:38.430556 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:38.430558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:38.432178 (+ 1620us) server_negotiation.cc:407] Connection header received
0809 08:56:38.432288 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:38.432289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:38.432301 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:38.432334 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:38.433161 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.433813 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.434808 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.435086 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.435132 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:38.436882 (+ 1750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:38.436884 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:38.436884 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:38.436891 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:38.438555 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:38.438941 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:38.438942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:38.438942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:38.439000 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:38.439233 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:38.439234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:38.439234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:38.439363 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:38.439451 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:38.439705 (+ 254us) server_negotiation.cc:299] Negotiation successful
0809 08:56:38.439789 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 08:56:38.448002 12132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:38.432226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51956 (local address 127.8.154.60:33545)
0809 08:56:38.432264 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:38.432266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:38.440480 (+ 8214us) server_negotiation.cc:407] Connection header received
0809 08:56:38.440587 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:38.440588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:38.440601 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:38.440634 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:38.441479 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.442271 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.443263 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.443552 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.443594 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:38.444983 (+ 1389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:38.444985 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:38.444985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:38.444993 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:38.446733 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:38.447151 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:38.447152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:38.447153 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:38.447212 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:38.447410 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:38.447411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:38.447411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:38.447540 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:38.447613 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:38.447826 (+ 213us) server_negotiation.cc:299] Negotiation successful
0809 08:56:38.447902 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 08:56:38.450681 12210 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:56:38.450913 12210 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:56:38.451346 12016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:38.431619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34384 (local address 127.8.154.61:40123)
0809 08:56:38.431809 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:38.431811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:38.435890 (+ 4079us) server_negotiation.cc:407] Connection header received
0809 08:56:38.435994 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:38.435995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:38.436006 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:38.436037 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:38.436702 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.437425 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.440048 (+ 2623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:38.440376 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:38.440424 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:38.448196 (+ 7772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:38.448198 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:38.448199 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:38.448208 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:38.449894 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:38.450315 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:38.450316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:38.450317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:38.450375 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:38.450588 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:38.450590 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:38.450590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:38.450728 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:38.450824 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:38.451178 (+ 354us) server_negotiation.cc:299] Negotiation successful
0809 08:56:38.451261 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165}
I20260809 08:56:38.461081 12089 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e4f23887a015480c8e9f0f7467d2b07c [term 1 FOLLOWER]: Refusing update from remote peer 9038a3d36e6d40b3b609218933b25dbe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:56:38.461103 11967 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 271cc2660f6d4ef9a4362ac8561403d9 [term 1 FOLLOWER]: Refusing update from remote peer 9038a3d36e6d40b3b609218933b25dbe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:56:38.461470 12193 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9038a3d36e6d40b3b609218933b25dbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4f23887a015480c8e9f0f7467d2b07c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 33545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:56:38.461673 12193 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9038a3d36e6d40b3b609218933b25dbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "271cc2660f6d4ef9a4362ac8561403d9" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 40123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:56:38.462999 12160 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4f23887a015480c8e9f0f7467d2b07c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9038a3d36e6d40b3b609218933b25dbe. Latest consensus state: current_term: 1 leader_uuid: "9038a3d36e6d40b3b609218933b25dbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9038a3d36e6d40b3b609218933b25dbe" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 33505 } } peers { permanent_uuid: "271cc2660f6d4ef9a4362ac8561403d9" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 40123 } } peers { permanent_uuid: "e4f23887a015480c8e9f0f7467d2b07c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 33545 } } }
I20260809 08:56:38.463095 12160 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4f23887a015480c8e9f0f7467d2b07c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:56:38.464885 12135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 271cc2660f6d4ef9a4362ac8561403d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9038a3d36e6d40b3b609218933b25dbe. Latest consensus state: current_term: 1 leader_uuid: "9038a3d36e6d40b3b609218933b25dbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9038a3d36e6d40b3b609218933b25dbe" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 33505 } } peers { permanent_uuid: "271cc2660f6d4ef9a4362ac8561403d9" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 40123 } } peers { permanent_uuid: "e4f23887a015480c8e9f0f7467d2b07c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 33545 } } }
I20260809 08:56:38.464978 12135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 271cc2660f6d4ef9a4362ac8561403d9 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 08:56:38.467958 11930 tablet.cc:2369] T 00000000000000000000000000000000 P 271cc2660f6d4ef9a4362ac8561403d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:56:38.469349 12193 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9038a3d36e6d40b3b609218933b25dbe [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9038a3d36e6d40b3b609218933b25dbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9038a3d36e6d40b3b609218933b25dbe" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 33505 } } peers { permanent_uuid: "271cc2660f6d4ef9a4362ac8561403d9" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 40123 } } peers { permanent_uuid: "e4f23887a015480c8e9f0f7467d2b07c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 33545 } } }
I20260809 08:56:38.469451 12193 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9038a3d36e6d40b3b609218933b25dbe [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:38.470110 12135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 271cc2660f6d4ef9a4362ac8561403d9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9038a3d36e6d40b3b609218933b25dbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9038a3d36e6d40b3b609218933b25dbe" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 33505 } } peers { permanent_uuid: "271cc2660f6d4ef9a4362ac8561403d9" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 40123 } } peers { permanent_uuid: "e4f23887a015480c8e9f0f7467d2b07c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 33545 } } }
I20260809 08:56:38.470191 12135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 271cc2660f6d4ef9a4362ac8561403d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:56:38.469379 12215 mvcc.cc:205] Tried to move back new op lower bound from 7316544710493880320 to 7316544710362587136. Current Snapshot: MvccSnapshot[applied={T|T < 7316544710493880320}]
I20260809 08:56:38.472496 12183 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9038a3d36e6d40b3b609218933b25dbe [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9038a3d36e6d40b3b609218933b25dbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9038a3d36e6d40b3b609218933b25dbe" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 33505 } } peers { permanent_uuid: "271cc2660f6d4ef9a4362ac8561403d9" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 40123 } } peers { permanent_uuid: "e4f23887a015480c8e9f0f7467d2b07c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 33545 } } }
I20260809 08:56:38.472579 12183 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9038a3d36e6d40b3b609218933b25dbe [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:38.473078 12160 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4f23887a015480c8e9f0f7467d2b07c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9038a3d36e6d40b3b609218933b25dbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9038a3d36e6d40b3b609218933b25dbe" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 33505 } } peers { permanent_uuid: "271cc2660f6d4ef9a4362ac8561403d9" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 40123 } } peers { permanent_uuid: "e4f23887a015480c8e9f0f7467d2b07c" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 33545 } } }
I20260809 08:56:38.473163 12160 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4f23887a015480c8e9f0f7467d2b07c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:56:38.473320 12216 mvcc.cc:205] Tried to move back new op lower bound from 7316544710493880320 to 7316544710362587136. Current Snapshot: MvccSnapshot[applied={T|T < 7316544710493880320}]
I20260809 08:56:38.473821 12210 catalog_manager.cc:1347] Generated new cluster ID: a4c15ca08906481fbce35440980076cb
I20260809 08:56:38.473875 12210 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:56:38.490456 12210 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:56:38.491199 12210 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:56:38.507493 12210 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9038a3d36e6d40b3b609218933b25dbe: Generated new TSK 0
I20260809 08:56:38.507704 12210 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 08:56:38.534430 12016 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:56:38.532508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34388 (local address 127.8.154.61:40123)
0809 08:56:38.532565 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:38.532567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:38.533510 (+ 943us) server_negotiation.cc:407] Connection header received
0809 08:56:38.533741 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:38.533743 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:38.533756 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:38.533788 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:38.534364 (+ 576us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34388: BlockingRecv error: recv got EOF from 127.0.0.1:34388 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260809 08:56:38.535157 11889 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:56:38.532785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47120 (local address 127.8.154.62:33505)
0809 08:56:38.532824 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:38.532825 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:38.533914 (+ 1089us) server_negotiation.cc:407] Connection header received
0809 08:56:38.534672 (+ 758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:38.534674 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:38.534684 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:38.534714 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:38.535089 (+ 375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47120: BlockingRecv error: recv got EOF from 127.0.0.1:47120 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 08:56:38.535986 12132 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:56:38.533145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51960 (local address 127.8.154.60:33545)
0809 08:56:38.533187 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:38.533189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:38.535423 (+ 2234us) server_negotiation.cc:407] Connection header received
0809 08:56:38.535552 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:38.535553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:38.535565 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:38.535595 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:38.535925 (+ 330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51960: BlockingRecv error: recv got EOF from 127.0.0.1:51960 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 08:56:38.538599 12016 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:56:38.537768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34400 (local address 127.8.154.61:40123)
0809 08:56:38.537802 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:38.537804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:38.538046 (+ 242us) server_negotiation.cc:407] Connection header received
0809 08:56:38.538158 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:38.538159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:38.538177 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:38.538209 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:38.538551 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34400: BlockingRecv error: recv got EOF from 127.0.0.1:34400 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 08:56:38.545416 12016 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:56:38.544590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34402 (local address 127.8.154.61:40123)
0809 08:56:38.544625 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:38.544627 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:38.544870 (+ 243us) server_negotiation.cc:407] Connection header received
0809 08:56:38.544973 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:38.544974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:38.544987 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:38.545017 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:38.545366 (+ 349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34402: BlockingRecv error: recv got EOF from 127.0.0.1:34402 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 08:56:38.549433 12016 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:56:38.548650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34406 (local address 127.8.154.61:40123)
0809 08:56:38.548684 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:38.548685 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:38.548907 (+ 222us) server_negotiation.cc:407] Connection header received
0809 08:56:38.549011 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:38.549012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:38.549023 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:38.549055 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:38.549385 (+ 330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34406: BlockingRecv error: recv got EOF from 127.0.0.1:34406 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 08:56:38.555585 12016 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:56:38.554672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34412 (local address 127.8.154.61:40123)
0809 08:56:38.554710 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:38.554711 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:38.554950 (+ 239us) server_negotiation.cc:407] Connection header received
0809 08:56:38.555055 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:38.555056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:38.555067 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:38.555099 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:38.555536 (+ 437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34412: BlockingRecv error: recv got EOF from 127.0.0.1:34412 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 08:56:38.566010 12016 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:56:38.565048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34422 (local address 127.8.154.61:40123)
0809 08:56:38.565094 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:38.565096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:38.565358 (+ 262us) server_negotiation.cc:407] Connection header received
0809 08:56:38.565467 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:38.565468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:38.565481 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:38.565512 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:38.565951 (+ 439us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34422: BlockingRecv error: recv got EOF from 127.0.0.1:34422 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 08:56:38.584463 12016 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:56:38.583457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34438 (local address 127.8.154.61:40123)
0809 08:56:38.583501 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:38.583503 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:38.583783 (+ 280us) server_negotiation.cc:407] Connection header received
0809 08:56:38.583898 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:38.583899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:38.583912 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:38.583946 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:38.584406 (+ 460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34438: BlockingRecv error: recv got EOF from 127.0.0.1:34438 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 08:56:38.618610 12016 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:56:38.617903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34452 (local address 127.8.154.61:40123)
0809 08:56:38.618003 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:38.618005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:38.618076 (+ 71us) server_negotiation.cc:407] Connection header received
0809 08:56:38.618122 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:38.618123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:38.618134 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:38.618208 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:38.618537 (+ 329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34452: BlockingRecv error: recv got EOF from 127.0.0.1:34452 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260809 08:56:38.684795 12016 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:56:38.684068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34456 (local address 127.8.154.61:40123)
0809 08:56:38.684172 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:38.684174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:38.684213 (+ 39us) server_negotiation.cc:407] Connection header received
0809 08:56:38.684290 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:38.684291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:38.684301 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:38.684359 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:38.684722 (+ 363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34456: BlockingRecv error: recv got EOF from 127.0.0.1:34456 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260809 08:56:38.814790 12016 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:56:38.814007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34460 (local address 127.8.154.61:40123)
0809 08:56:38.814087 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:38.814090 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:38.814219 (+ 129us) server_negotiation.cc:407] Connection header received
0809 08:56:38.814267 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:38.814268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:38.814279 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:38.814340 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:38.814703 (+ 363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34460: BlockingRecv error: recv got EOF from 127.0.0.1:34460 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260809 08:56:39.072845 12016 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:56:39.072115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34470 (local address 127.8.154.61:40123)
0809 08:56:39.072181 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:39.072183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:39.072240 (+ 57us) server_negotiation.cc:407] Connection header received
0809 08:56:39.072333 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:39.072334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:39.072350 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:39.072414 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:39.072793 (+ 379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34470: BlockingRecv error: recv got EOF from 127.0.0.1:34470 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 08:56:39.389838 12153 catalog_manager.cc:1259] Loaded cluster ID: a4c15ca08906481fbce35440980076cb
I20260809 08:56:39.389910 12153 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 271cc2660f6d4ef9a4362ac8561403d9: loading cluster ID for follower catalog manager: success
I20260809 08:56:39.390753 12153 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 271cc2660f6d4ef9a4362ac8561403d9: acquiring CA information for follower catalog manager: success
I20260809 08:56:39.391090 12153 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 271cc2660f6d4ef9a4362ac8561403d9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 08:56:39.413443 12180 catalog_manager.cc:1259] Loaded cluster ID: a4c15ca08906481fbce35440980076cb
I20260809 08:56:39.413527 12180 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e4f23887a015480c8e9f0f7467d2b07c: loading cluster ID for follower catalog manager: success
I20260809 08:56:39.414412 12180 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e4f23887a015480c8e9f0f7467d2b07c: acquiring CA information for follower catalog manager: success
I20260809 08:56:39.414788 12180 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e4f23887a015480c8e9f0f7467d2b07c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 08:56:39.589071 12344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:39.586230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39850 (local address 127.8.154.61:40123)
0809 08:56:39.586355 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:39.586357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:39.586373 (+ 16us) server_negotiation.cc:407] Connection header received
0809 08:56:39.586441 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:39.586442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:39.586495 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:39.586580 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:56:39.586727 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:39.587311 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:39.588131 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:39.588400 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:39.588461 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:39.588656 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:56:39.588772 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:56:39.588835 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:39.588909 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 08:56:39.588929 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":71,"threads_started":1}
I20260809 08:56:39.593465 12357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:39.591156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46142 (local address 127.8.154.60:33545)
0809 08:56:39.591266 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:39.591267 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:39.591276 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:56:39.591315 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:39.591316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:39.591353 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:39.591423 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:56:39.591525 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:39.591992 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:39.592740 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:39.592931 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:39.592995 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:39.593129 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:56:39.593230 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:56:39.593290 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:39.593330 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 08:56:39.593346 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":60,"threads_started":1}
I20260809 08:56:39.598210 12344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:39.595242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39866 (local address 127.8.154.61:40123)
0809 08:56:39.595380 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:39.595383 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:39.595508 (+ 125us) server_negotiation.cc:407] Connection header received
0809 08:56:39.595619 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:39.595621 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:39.595633 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:39.595664 (+ 31us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:56:39.595838 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:39.596400 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:39.597382 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:39.597690 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:39.597748 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:39.597917 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:56:39.597949 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:56:39.597977 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:39.598092 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 08:56:39.598117 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260809 08:56:39.598591 12364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:39.595257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59076 (local address 127.8.154.62:33505)
0809 08:56:39.595403 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:39.595405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:39.595414 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:56:39.595460 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:39.595461 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:39.595515 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:39.595617 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:56:39.595875 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:39.596698 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:39.597652 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:39.597859 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:39.597937 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:39.598133 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:56:39.598253 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:56:39.598313 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:39.598485 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 08:56:39.598511 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":73,"threads_started":1}
I20260809 08:56:39.598694 12357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:39.595622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46156 (local address 127.8.154.60:33545)
0809 08:56:39.595660 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:39.595662 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:39.595731 (+ 69us) server_negotiation.cc:407] Connection header received
0809 08:56:39.595776 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:39.595777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:39.595790 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:39.595852 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:56:39.596018 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:39.596744 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:39.597661 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:39.597943 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:39.597984 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:39.598199 (+ 215us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:56:39.598232 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:56:39.598291 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:39.598580 (+ 289us) server_negotiation.cc:299] Negotiation successful
0809 08:56:39.598609 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 08:56:39.600335 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 11756
I20260809 08:56:39.618250 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 11897
I20260809 08:56:39.629494 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 12018
2026-08-09T08:56:39Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3108 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:56:39 dist-test-slave-t3q3 krb5kdc[12373](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:56:39 dist-test-slave-t3q3 krb5kdc[12373](info): set up 2 sockets
Aug 09 08:56:39 dist-test-slave-t3q3 krb5kdc[12373](info): commencing operation
krb5kdc: starting...
W20260809 08:56:40.901504 8808 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/krb5kdc/test-user.keytab.
Aug 09 08:56:40 dist-test-slave-t3q3 krb5kdc[12373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:56:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:56:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.8.154.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
Entry for principal kudu/127.8.154.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab.
I20260809 08:56:41.025115 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:39313
--webserver_interface=127.8.154.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:41549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:39313,127.8.154.61:35915,127.8.154.60:38091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:56:41.109109 12390 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 08:56:41.109354 12390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:41.109373 12390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:41.109395 12390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:41.110683 12390 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:56:41.110731 12390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:41.110746 12390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:41.110759 12390 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:56:41.110771 12390 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:56:41.112437 12390 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:41549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:39313,127.8.154.61:35915,127.8.154.60:38091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.62.keytab
--principal=kudu/127.8.154.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.62:39313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.8.154.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:41.112726 12390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:41.112974 12390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:41.118052 12398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:41.118147 12390 server_base.cc:1034] running on GCE node
W20260809 08:56:41.118062 12405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:41.118062 12396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:41.118647 12390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:41.118935 12390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:41.120134 12390 hybrid_clock.cc:648] HybridClock initialized: now 1786265801120112 us; error 36 us; skew 500 ppm
Aug 09 08:56:41 dist-test-slave-t3q3 krb5kdc[12373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265801, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:56:41.122861 12390 init.cc:377] Logged in from keytab as kudu/127.8.154.62@KRBTEST.COM (short username kudu)
I20260809 08:56:41.123245 12390 webserver.cc:460] Webserver started at http://127.8.154.62:43193/ using document root <none> and password file <none>
I20260809 08:56:41.123445 12390 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:41.123504 12390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:41.123602 12390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:41.124713 12390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-0/data/instance:
uuid: "af05a807602e41d3b424447c06734947"
format_stamp: "Formatted at 2026-08-09 08:56:41 on dist-test-slave-t3q3"
server_key: "ad86d6915444111857f3e504d0c353c4"
server_key_iv: "6f200d48a0569502d9e59f65fef83160"
server_key_version: "encryptionkey@0"
I20260809 08:56:41.125038 12390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-0/wal/instance:
uuid: "af05a807602e41d3b424447c06734947"
format_stamp: "Formatted at 2026-08-09 08:56:41 on dist-test-slave-t3q3"
server_key: "ad86d6915444111857f3e504d0c353c4"
server_key_iv: "6f200d48a0569502d9e59f65fef83160"
server_key_version: "encryptionkey@0"
I20260809 08:56:41.126405 12390 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:56:41.127224 12417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:41.127426 12390 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:56:41.127555 12390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-0/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-0/wal
uuid: "af05a807602e41d3b424447c06734947"
format_stamp: "Formatted at 2026-08-09 08:56:41 on dist-test-slave-t3q3"
server_key: "ad86d6915444111857f3e504d0c353c4"
server_key_iv: "6f200d48a0569502d9e59f65fef83160"
server_key_version: "encryptionkey@0"
I20260809 08:56:41.127643 12390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:41.137600 12390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:41.139853 12390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:41.140024 12390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:41.143915 12390 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.62:39313
I20260809 08:56:41.143950 12518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.62:39313 every 8 connection(s)
I20260809 08:56:41.144380 12390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-0/data/info.pb
I20260809 08:56:41.145437 12519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:41.147048 12519 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.154.62" port: 39313 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:56:41.150525 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 12390
I20260809 08:56:41.150635 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-0/wal/instance
I20260809 08:56:41.150851 8808 external_mini_cluster.cc:1428] Setting key 87acfcbb7e6e3b327dd9cf2efae979ee
Aug 09 08:56:41 dist-test-slave-t3q3 krb5kdc[12373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265801, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:56:41.157815 12521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:41.147770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:39313 (local address 127.0.0.1:52970)
0809 08:56:41.147951 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:41.147999 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:41.148520 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:41.148816 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:41.148819 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:41.149066 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:56:41.149230 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:41.149234 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.150256 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.150257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:41.151389 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:41.151392 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.151939 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.151941 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:41.152029 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:41.152582 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:56:41.152594 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:56:41.154335 (+ 1741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:56:41.156247 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:41.156253 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:41.156255 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:41.156466 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:41.156736 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:41.156737 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:41.156738 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:41.156832 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:41.157208 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:56:41.157213 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:56:41.157279 (+ 66us) client_negotiation.cc:769] Sending connection context
0809 08:56:41.157388 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 08:56:41.157630 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":99,"thread_start_us":86,"threads_started":1}
I20260809 08:56:41.157819 12525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:41.147813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52970 (local address 127.8.154.62:39313)
0809 08:56:41.148015 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:41.148019 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:41.148112 (+ 93us) server_negotiation.cc:407] Connection header received
0809 08:56:41.148684 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:41.148686 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:41.148703 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:41.148787 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:41.149318 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.150165 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.151493 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.151825 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.151922 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:41.154441 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:41.154450 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:41.154454 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:41.154461 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:41.156151 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:41.156566 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:41.156569 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:41.156569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:41.156664 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:41.156909 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:41.156909 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:41.156910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:41.157129 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:41.157216 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:41.157423 (+ 207us) server_negotiation.cc:299] Negotiation successful
0809 08:56:41.157564 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":111,"threads_started":1}
I20260809 08:56:41.158638 12519 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35915 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:56:41.158983 12521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:41.158871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.61:35915 (local address 127.0.0.1:33878)
0809 08:56:41.158920 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:41.158963 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.61:35915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 08:56:41.159121 12419 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.154.61:35915: connect: Connection refused (error 111)
I20260809 08:56:41.159540 12521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:41.159449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.61:35915 (local address 127.0.0.1:33894)
0809 08:56:41.159494 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:41.159518 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.61:35915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 08:56:41.159657 12519 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.154.61:35915: Network error: Client connection negotiation failed: client connection to 127.8.154.61:35915: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.8.154.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.61.keytab.
Entry for principal kudu/127.8.154.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.61.keytab.
I20260809 08:56:41.178560 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.61:35915
--webserver_interface=127.8.154.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.61.keytab
--principal=kudu/127.8.154.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:41549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:39313,127.8.154.61:35915,127.8.154.60:38091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:56:41.208894 12519 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35915 } attempt: 1
I20260809 08:56:41.209420 12521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:41.209277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.61:35915 (local address 127.0.0.1:33904)
0809 08:56:41.209346 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:41.209389 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.61:35915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 08:56:41.209896 12521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:41.209796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.61:35915 (local address 127.0.0.1:33908)
0809 08:56:41.209849 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:41.209878 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.61:35915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 08:56:41.209996 12519 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.154.61:35915: Network error: Client connection negotiation failed: client connection to 127.8.154.61:35915: connect: Connection refused (error 111)
W20260809 08:56:41.264683 12530 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 08:56:41.264883 12530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:41.264901 12530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:41.264950 12530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:41.266064 12530 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:56:41.266104 12530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:41.266119 12530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:41.266131 12530 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:56:41.266170 12530 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:56:41.267657 12530 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:41549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:39313,127.8.154.61:35915,127.8.154.60:38091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.61.keytab
--principal=kudu/127.8.154.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.61:35915
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.8.154.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:41.267911 12530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:41.268201 12530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:41.273095 12549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:41.273129 12551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:41.273175 12545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:41.273334 12530 server_base.cc:1034] running on GCE node
I20260809 08:56:41.273600 12530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:41.273909 12530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:41.275090 12530 hybrid_clock.cc:648] HybridClock initialized: now 1786265801275072 us; error 28 us; skew 500 ppm
Aug 09 08:56:41 dist-test-slave-t3q3 krb5kdc[12373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265801, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:56:41.277735 12530 init.cc:377] Logged in from keytab as kudu/127.8.154.61@KRBTEST.COM (short username kudu)
I20260809 08:56:41.278179 12530 webserver.cc:460] Webserver started at http://127.8.154.61:43673/ using document root <none> and password file <none>
I20260809 08:56:41.278168 12519 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35915 } attempt: 2
I20260809 08:56:41.278491 12530 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:41.278569 12530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:41.278659 12521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:41.278503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.61:35915 (local address 127.0.0.1:33918)
0809 08:56:41.278582 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:41.278621 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.61:35915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 08:56:41.278736 12530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:41.279017 12521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:41.278955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.61:35915 (local address 127.0.0.1:33922)
0809 08:56:41.278981 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:41.279002 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.61:35915: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260809 08:56:41.279127 12519 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.154.61:35915: Network error: Client connection negotiation failed: client connection to 127.8.154.61:35915: connect: Connection refused (error 111)
I20260809 08:56:41.279920 12530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-1/data/instance:
uuid: "21af4c94d4ad4aafa50f8526e9663bea"
format_stamp: "Formatted at 2026-08-09 08:56:41 on dist-test-slave-t3q3"
server_key: "912534dfc20899b1753bfebba92588be"
server_key_iv: "74f3b0e205749ce03c3bf53352ce9e90"
server_key_version: "encryptionkey@0"
I20260809 08:56:41.280355 12530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-1/wal/instance:
uuid: "21af4c94d4ad4aafa50f8526e9663bea"
format_stamp: "Formatted at 2026-08-09 08:56:41 on dist-test-slave-t3q3"
server_key: "912534dfc20899b1753bfebba92588be"
server_key_iv: "74f3b0e205749ce03c3bf53352ce9e90"
server_key_version: "encryptionkey@0"
I20260809 08:56:41.281726 12530 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:56:41.282461 12562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:41.282642 12530 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:56:41.282716 12530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-1/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-1/wal
uuid: "21af4c94d4ad4aafa50f8526e9663bea"
format_stamp: "Formatted at 2026-08-09 08:56:41 on dist-test-slave-t3q3"
server_key: "912534dfc20899b1753bfebba92588be"
server_key_iv: "74f3b0e205749ce03c3bf53352ce9e90"
server_key_version: "encryptionkey@0"
I20260809 08:56:41.282809 12530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:41.305373 12530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:41.307802 12530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:41.307965 12530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:41.312043 12530 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.61:35915
I20260809 08:56:41.312184 12655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.61:35915 every 8 connection(s)
I20260809 08:56:41.312515 12530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-1/data/info.pb
I20260809 08:56:41.313589 12660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:41.314174 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 12530
I20260809 08:56:41.314291 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-1/wal/instance
I20260809 08:56:41.314514 8808 external_mini_cluster.cc:1428] Setting key bb0f1ef5e822b39b5f11d491830fa294
I20260809 08:56:41.316188 12660 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.154.62" port: 39313 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 08:56:41 dist-test-slave-t3q3 krb5kdc[12373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265801, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.61@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.8.154.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.8.154.60@KRBTEST.COM" created.
I20260809 08:56:41.326224 12525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:41.317149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52982 (local address 127.8.154.62:39313)
0809 08:56:41.317208 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:41.317210 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:41.317429 (+ 219us) server_negotiation.cc:407] Connection header received
0809 08:56:41.317968 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:41.317970 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:41.317989 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:41.318041 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:41.318674 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.319269 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.320360 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.320604 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.320689 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:41.323287 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:41.323288 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:41.323288 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:41.323296 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:41.324734 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:41.325210 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:41.325211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:41.325212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:41.325285 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:41.325527 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:41.325527 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:41.325528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:41.325640 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:41.325693 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:41.326075 (+ 382us) server_negotiation.cc:299] Negotiation successful
0809 08:56:41.326156 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 08:56:41.326488 12663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:41.317126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:39313 (local address 127.0.0.1:52982)
0809 08:56:41.317298 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:41.317336 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:41.317846 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:41.318123 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:41.318126 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:41.318376 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:56:41.318571 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:41.318574 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.319385 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.319386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:41.320268 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:41.320270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.320711 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.320712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:41.320845 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:41.321418 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:56:41.321426 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:56:41.323186 (+ 1760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:56:41.324821 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:41.324828 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:41.324830 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:41.325105 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:41.325347 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:41.325348 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:41.325348 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:41.325463 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:41.325705 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:56:41.325711 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:56:41.325913 (+ 202us) client_negotiation.cc:769] Sending connection context
0809 08:56:41.326040 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 08:56:41.326234 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":90,"threads_started":1}
I20260809 08:56:41.327050 12660 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35915 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 08:56:41 dist-test-slave-t3q3 krb5kdc[12373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265801, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.61@KRBTEST.COM for kudu/127.8.154.61@KRBTEST.COM
I20260809 08:56:41.334589 12663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:41.327347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.61:35915 (local address 127.0.0.1:33924)
0809 08:56:41.327538 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:41.327548 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:41.327593 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:41.327744 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:41.327744 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:41.327884 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:56:41.327929 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:41.327930 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.328806 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.328807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:41.329314 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:41.329316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.329829 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.329830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:41.329898 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:41.330355 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:56:41.330361 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:56:41.331636 (+ 1275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:56:41.333494 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:41.333495 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:41.333495 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:41.333670 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:41.333903 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:41.333904 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:41.333904 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:41.333969 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:41.334261 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:56:41.334262 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:56:41.334300 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 08:56:41.334365 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 08:56:41.334465 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175}
I20260809 08:56:41.334662 12664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:41.327394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33924 (local address 127.8.154.61:35915)
0809 08:56:41.327561 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:41.327566 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:41.327591 (+ 25us) server_negotiation.cc:407] Connection header received
0809 08:56:41.327651 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:41.327654 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:41.327666 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:41.327736 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:41.328014 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.328714 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.329414 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.329739 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.329819 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:41.331736 (+ 1917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:41.331744 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:41.331744 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:41.331752 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:41.333387 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:41.333766 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:41.333769 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:41.333770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:41.333829 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:41.334040 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:41.334041 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:41.334041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:41.334185 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:41.334281 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:41.334394 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 08:56:41.334534 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":127,"thread_start_us":95,"threads_started":1}
I20260809 08:56:41.335317 12660 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.154.60" port: 38091 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:56:41.335661 12663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:41.335529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:38091 (local address 127.0.0.1:39416)
0809 08:56:41.335590 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:41.335643 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.60:38091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 08:56:41.335803 12566 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.8.154.60:38091: connect: Connection refused (error 111)
I20260809 08:56:41.336318 12663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:41.336216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:38091 (local address 127.0.0.1:39426)
0809 08:56:41.336275 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:41.336300 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.60:38091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 08:56:41.336453 12660 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.154.60:38091: Network error: Client connection negotiation failed: client connection to 127.8.154.60:38091: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.8.154.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.60.keytab.
Entry for principal kudu/127.8.154.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.60.keytab.
I20260809 08:56:41.340972 8808 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskBbavBz/build/release/bin/kudu
/tmp/dist-test-taskBbavBz/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.60:38091
--webserver_interface=127.8.154.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.60.keytab
--principal=kudu/127.8.154.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.8.154.20:41549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.8.154.62:39313,127.8.154.61:35915,127.8.154.60:38091
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:56:41.370301 12519 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35915 } attempt: 3
Aug 09 08:56:41 dist-test-slave-t3q3 krb5kdc[12373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265801, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for kudu/127.8.154.61@KRBTEST.COM
I20260809 08:56:41.377693 12664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:41.370812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33934 (local address 127.8.154.61:35915)
0809 08:56:41.370921 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:41.370923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:41.370933 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:56:41.370964 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:41.370964 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:41.370984 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:41.371056 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:41.371410 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.371918 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.372758 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.373003 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.373085 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:41.375019 (+ 1934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:41.375020 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:41.375021 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:41.375029 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:41.376530 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:41.376878 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:41.376879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:41.376880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:41.376941 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:41.377153 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:41.377154 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:41.377154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:41.377269 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:41.377342 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:41.377525 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 08:56:41.377602 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 08:56:41.377693 12521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:41.370730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.61:35915 (local address 127.0.0.1:33934)
0809 08:56:41.370788 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:41.370805 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:41.370883 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:41.371064 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:41.371066 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:41.371247 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:56:41.371316 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:41.371318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.372015 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.372016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:41.372652 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:41.372654 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.373084 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.373085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:41.373175 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:41.373661 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:56:41.373666 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:56:41.374903 (+ 1237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:56:41.376621 (+ 1718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:41.376622 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:41.376623 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:41.376802 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:41.377013 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:41.377014 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:41.377014 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:41.377082 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:41.377346 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:56:41.377347 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:56:41.377397 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 08:56:41.377477 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 08:56:41.377588 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 08:56:41.378384 12519 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.154.60" port: 38091 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:56:41.378790 12521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:41.378664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:38091 (local address 127.0.0.1:39432)
0809 08:56:41.378719 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:41.378759 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.60:38091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 08:56:41.379189 12521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:41.379088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:38091 (local address 127.0.0.1:39448)
0809 08:56:41.379135 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:41.379166 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.60:38091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 08:56:41.379324 12519 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.154.60:38091: Network error: Client connection negotiation failed: client connection to 127.8.154.60:38091: connect: Connection refused (error 111)
I20260809 08:56:41.385705 12660 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.154.60" port: 38091 } attempt: 1
I20260809 08:56:41.386109 12663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:41.385997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:38091 (local address 127.0.0.1:39454)
0809 08:56:41.386052 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:41.386085 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.60:38091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 08:56:41.386525 12663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:41.386376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:38091 (local address 127.0.0.1:39462)
0809 08:56:41.386426 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:41.386497 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.60:38091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 08:56:41.386641 12660 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.154.60:38091: Network error: Client connection negotiation failed: client connection to 127.8.154.60:38091: connect: Connection refused (error 111)
I20260809 08:56:41.410516 12519 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.154.60" port: 38091 } attempt: 1
I20260809 08:56:41.411034 12521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:41.410858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:38091 (local address 127.0.0.1:39472)
0809 08:56:41.410934 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:41.410987 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.60:38091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 08:56:41.411474 12521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:41.411374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:38091 (local address 127.0.0.1:39476)
0809 08:56:41.411424 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:41.411452 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.60:38091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 08:56:41.411636 12519 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.154.60:38091: Network error: Client connection negotiation failed: client connection to 127.8.154.60:38091: connect: Connection refused (error 111)
W20260809 08:56:41.428158 12670 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 08:56:41.428367 12670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:56:41.428385 12670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:56:41.428428 12670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:56:41.429559 12670 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:56:41.429600 12670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:56:41.429615 12670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:56:41.429627 12670 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:56:41.429675 12670 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:56:41.431325 12670 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.8.154.20:41549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.8.154.62:39313,127.8.154.61:35915,127.8.154.60:38091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/krb5kdc/kudu_127.8.154.60.keytab
--principal=kudu/127.8.154.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.8.154.60:38091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.8.154.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12670
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:56:41.431584 12670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:56:41.431843 12670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:56:41.436887 12682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:41.436895 12681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:56:41.436896 12688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:56:41.437253 12670 server_base.cc:1034] running on GCE node
I20260809 08:56:41.437510 12670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:56:41.437812 12670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:56:41.438956 12670 hybrid_clock.cc:648] HybridClock initialized: now 1786265801438945 us; error 31 us; skew 500 ppm
Aug 09 08:56:41 dist-test-slave-t3q3 krb5kdc[12373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265801, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:56:41.442008 12670 init.cc:377] Logged in from keytab as kudu/127.8.154.60@KRBTEST.COM (short username kudu)
I20260809 08:56:41.442482 12670 webserver.cc:460] Webserver started at http://127.8.154.60:38185/ using document root <none> and password file <none>
I20260809 08:56:41.442742 12670 fs_manager.cc:359] Metadata directory not provided
I20260809 08:56:41.442795 12670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:56:41.443020 12670 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:56:41.444192 12670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-2/data/instance:
uuid: "af915e0b56c0482bbbad5799b7b6e87e"
format_stamp: "Formatted at 2026-08-09 08:56:41 on dist-test-slave-t3q3"
server_key: "fc875d07c300f4702a24dde005c4bb13"
server_key_iv: "6e8f521fac97f4c0fd9b0336e7118998"
server_key_version: "encryptionkey@0"
I20260809 08:56:41.444562 12670 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-2/wal/instance:
uuid: "af915e0b56c0482bbbad5799b7b6e87e"
format_stamp: "Formatted at 2026-08-09 08:56:41 on dist-test-slave-t3q3"
server_key: "fc875d07c300f4702a24dde005c4bb13"
server_key_iv: "6e8f521fac97f4c0fd9b0336e7118998"
server_key_version: "encryptionkey@0"
I20260809 08:56:41.445966 12670 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:56:41.446790 12701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:56:41.447036 12670 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:56:41.447152 12670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-2/data,/tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-2/wal
uuid: "af915e0b56c0482bbbad5799b7b6e87e"
format_stamp: "Formatted at 2026-08-09 08:56:41 on dist-test-slave-t3q3"
server_key: "fc875d07c300f4702a24dde005c4bb13"
server_key_iv: "6e8f521fac97f4c0fd9b0336e7118998"
server_key_version: "encryptionkey@0"
I20260809 08:56:41.447232 12670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:56:41.454547 12670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:56:41.454782 12660 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.154.60" port: 38091 } attempt: 2
I20260809 08:56:41.455255 12663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:41.455091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:38091 (local address 127.0.0.1:39480)
0809 08:56:41.455174 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:41.455223 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.60:38091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 08:56:41.455606 12663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:41.455504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:38091 (local address 127.0.0.1:39494)
0809 08:56:41.455555 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:41.455587 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.8.154.60:38091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 08:56:41.455704 12660 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.8.154.60:38091: Network error: Client connection negotiation failed: client connection to 127.8.154.60:38091: connect: Connection refused (error 111)
I20260809 08:56:41.457067 12670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:56:41.457238 12670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:56:41.461099 12670 rpc_server.cc:307] RPC server started. Bound to: 127.8.154.60:38091
I20260809 08:56:41.461133 12782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.8.154.60:38091 every 8 connection(s)
I20260809 08:56:41.461522 12670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-2/data/info.pb
I20260809 08:56:41.462535 12783 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:56:41.464175 12783 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.154.62" port: 39313 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:56:41.466430 8808 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskBbavBz/build/release/bin/kudu as pid 12670
I20260809 08:56:41.466571 8808 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskBbavBz/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265519942966-8808-0/minicluster-data/master-2/wal/instance
I20260809 08:56:41.466789 8808 external_mini_cluster.cc:1428] Setting key d6ad772de92ade5a000ef7ca2fee9139
Aug 09 08:56:41 dist-test-slave-t3q3 krb5kdc[12373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265801, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.60@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
Aug 09 08:56:41 dist-test-slave-t3q3 krb5kdc[12373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.154.60@KRBTEST.COM
I20260809 08:56:41.474086 12525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:41.464869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52990 (local address 127.8.154.62:39313)
0809 08:56:41.464928 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:41.464930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:41.465145 (+ 215us) server_negotiation.cc:407] Connection header received
0809 08:56:41.465589 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:41.465590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:41.465607 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:41.465661 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:41.466278 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.467018 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.468397 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.468586 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.468650 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:41.470815 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:41.470817 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:41.470817 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:41.470826 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:41.472363 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:41.472859 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:41.472860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:41.472860 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:41.472929 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:41.473247 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:41.473248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:41.473248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:41.473434 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:41.473543 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:41.473839 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 08:56:41.473964 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 08:56:41.474164 12787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:41.464846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.62:39313 (local address 127.0.0.1:52990)
0809 08:56:41.465022 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:41.465059 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:41.465495 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:41.465730 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:41.465732 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:41.465968 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:56:41.466167 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:41.466171 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.467140 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.467141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:41.468313 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:41.468315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.468691 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.468692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:41.468833 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:41.469407 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:56:41.469415 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:56:41.470746 (+ 1331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:56:41.472449 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:41.472457 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:41.472458 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:41.472771 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:41.473034 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:41.473035 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:41.473035 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:41.473171 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:41.473533 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:56:41.473539 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:56:41.473662 (+ 123us) client_negotiation.cc:769] Sending connection context
0809 08:56:41.473791 (+ 129us) client_negotiation.cc:240] Negotiation successful
0809 08:56:41.473966 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":88,"threads_started":1}
I20260809 08:56:41.474814 12783 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35915 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:56:41.476140 12789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:41.467165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39496 (local address 127.8.154.60:38091)
0809 08:56:41.467360 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:41.467364 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:41.467409 (+ 45us) server_negotiation.cc:407] Connection header received
0809 08:56:41.467480 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:41.467483 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:41.467504 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:41.467581 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:41.468211 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.469138 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.470065 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.470377 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.470463 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:41.472864 (+ 2401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:41.472873 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:41.472874 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:41.472880 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:41.474820 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:41.475221 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:41.475223 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:41.475224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:41.475275 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:41.475491 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:41.475492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:41.475492 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:41.475677 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:41.475824 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:41.475918 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 08:56:41.476001 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":130,"threads_started":1}
Aug 09 08:56:41 dist-test-slave-t3q3 krb5kdc[12373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265801, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.60@KRBTEST.COM for kudu/127.8.154.61@KRBTEST.COM
Aug 09 08:56:41 dist-test-slave-t3q3 krb5kdc[12373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.154.62@KRBTEST.COM
I20260809 08:56:41.482254 12787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:41.475109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.61:35915 (local address 127.0.0.1:33938)
0809 08:56:41.475276 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:41.475285 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:41.475362 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:41.475537 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:41.475538 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:41.475646 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:56:41.475679 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:41.475681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.476573 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.476574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:41.477303 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:41.477306 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.477660 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.477661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:41.477729 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:41.478137 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:56:41.478141 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:56:41.479597 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:56:41.481223 (+ 1626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:41.481224 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:41.481224 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:41.481428 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:41.481645 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:41.481646 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:41.481646 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:41.481702 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:41.481999 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:56:41.482000 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:56:41.482049 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 08:56:41.482119 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 08:56:41.482191 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154}
I20260809 08:56:41.482366 12664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:41.475193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33938 (local address 127.8.154.61:35915)
0809 08:56:41.475251 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:41.475253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:41.475337 (+ 84us) server_negotiation.cc:407] Connection header received
0809 08:56:41.475436 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:41.475437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:41.475447 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:41.475521 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:41.475759 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.476483 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.477383 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.477583 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.477638 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:41.479684 (+ 2046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:41.479686 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:41.479686 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:41.479693 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:41.481124 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:41.481507 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:41.481509 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:41.481509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:41.481573 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:41.481770 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:41.481771 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:41.481772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:41.481904 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:41.481990 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:41.482172 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 08:56:41.482247 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 08:56:41.482751 12783 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.8.154.60" port: 38091 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:56:41.484452 12525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:41.476979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52994 (local address 127.8.154.62:39313)
0809 08:56:41.477024 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:41.477026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:41.477075 (+ 49us) server_negotiation.cc:407] Connection header received
0809 08:56:41.477144 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:41.477145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:41.477153 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:41.477196 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:41.477759 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.478184 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.479017 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.479332 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.479407 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:41.481557 (+ 2150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:41.481558 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:41.481559 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:41.481565 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:41.483277 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:41.483631 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:41.483632 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:41.483633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:41.483687 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:41.483906 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:41.483907 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:41.483907 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:41.484020 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:41.484242 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:41.484279 (+ 37us) server_negotiation.cc:299] Negotiation successful
0809 08:56:41.484352 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
Aug 09 08:56:41 dist-test-slave-t3q3 krb5kdc[12373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265801, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.60@KRBTEST.COM for kudu/127.8.154.60@KRBTEST.COM
I20260809 08:56:41.486801 12519 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.154.60" port: 38091 } attempt: 2
I20260809 08:56:41.489492 12787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:41.483038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:38091 (local address 127.0.0.1:39500)
0809 08:56:41.483069 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:41.483078 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:41.483117 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:41.483238 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:41.483239 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:41.483368 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:56:41.483392 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:41.483393 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.483988 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.483989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:41.484585 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:41.484587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.485029 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.485030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:41.485109 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:41.485532 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:56:41.485537 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:56:41.486687 (+ 1150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:56:41.488428 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:41.488429 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:41.488429 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:41.488600 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:41.488803 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:41.488804 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:41.488805 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:41.488860 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:41.489127 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:56:41.489128 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:56:41.489183 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 08:56:41.489258 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 08:56:41.489357 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16}
I20260809 08:56:41.489552 12789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:41.483076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39500 (local address 127.8.154.60:38091)
0809 08:56:41.483110 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:41.483113 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:41.483120 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:56:41.483173 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:41.483174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:41.483183 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:41.483243 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:41.483467 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.483909 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.484666 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.484946 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.485016 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:41.486784 (+ 1768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:41.486785 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:41.486786 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:41.486794 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:41.488350 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:41.488675 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:41.488676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:41.488676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:41.488728 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:41.488932 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:41.488933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:41.488933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:41.489048 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:41.489114 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:41.489303 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 08:56:41.489415 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":19}
Aug 09 08:56:41 dist-test-slave-t3q3 krb5kdc[12373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265800, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.8.154.61@KRBTEST.COM
I20260809 08:56:41.491349 12783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af915e0b56c0482bbbad5799b7b6e87e: Bootstrap starting.
I20260809 08:56:41.491943 12783 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af915e0b56c0482bbbad5799b7b6e87e: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:41.492352 12783 log.cc:824] T 00000000000000000000000000000000 P af915e0b56c0482bbbad5799b7b6e87e: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:41.493283 12783 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af915e0b56c0482bbbad5799b7b6e87e: No bootstrap required, opened a new log
I20260809 08:56:41.493469 12664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:41.485134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33940 (local address 127.8.154.61:35915)
0809 08:56:41.485171 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:41.485172 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:41.485180 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:56:41.485201 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:41.485202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:41.485212 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:41.485269 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:41.485917 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.486532 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.487467 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.487768 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.487830 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:41.490454 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:41.490455 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:41.490455 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:41.490462 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:41.491943 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:41.492415 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:41.492416 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:41.492416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:41.492479 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:41.492711 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:41.492712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:41.492713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:41.492855 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:41.493013 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:41.493274 (+ 261us) server_negotiation.cc:299] Negotiation successful
0809 08:56:41.493358 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Aug 09 08:56:41 dist-test-slave-t3q3 krb5kdc[12373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265801, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.62@KRBTEST.COM for kudu/127.8.154.60@KRBTEST.COM
I20260809 08:56:41.495559 8808 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:56:41.495545 12783 raft_consensus.cc:348] T 00000000000000000000000000000000 P af915e0b56c0482bbbad5799b7b6e87e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af05a807602e41d3b424447c06734947" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 39313 } } peers { permanent_uuid: "21af4c94d4ad4aafa50f8526e9663bea" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35915 } } peers { permanent_uuid: "af915e0b56c0482bbbad5799b7b6e87e" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 38091 } }
I20260809 08:56:41.495726 12783 raft_consensus.cc:729] T 00000000000000000000000000000000 P af915e0b56c0482bbbad5799b7b6e87e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af915e0b56c0482bbbad5799b7b6e87e, State: Initialized, Role: FOLLOWER
I20260809 08:56:41.495819 12783 consensus_queue.cc:260] T 00000000000000000000000000000000 P af915e0b56c0482bbbad5799b7b6e87e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af05a807602e41d3b424447c06734947" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 39313 } } peers { permanent_uuid: "21af4c94d4ad4aafa50f8526e9663bea" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35915 } } peers { permanent_uuid: "af915e0b56c0482bbbad5799b7b6e87e" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 38091 } }
I20260809 08:56:41.496091 12794 sys_catalog.cc:455] T 00000000000000000000000000000000 P af915e0b56c0482bbbad5799b7b6e87e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af05a807602e41d3b424447c06734947" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 39313 } } peers { permanent_uuid: "21af4c94d4ad4aafa50f8526e9663bea" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35915 } } peers { permanent_uuid: "af915e0b56c0482bbbad5799b7b6e87e" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 38091 } } }
I20260809 08:56:41.496155 12783 sys_catalog.cc:564] T 00000000000000000000000000000000 P af915e0b56c0482bbbad5799b7b6e87e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:56:41.496187 12794 sys_catalog.cc:458] T 00000000000000000000000000000000 P af915e0b56c0482bbbad5799b7b6e87e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:56:41.497092 12521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:41.487033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:38091 (local address 127.0.0.1:39506)
0809 08:56:41.487238 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:41.487249 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:41.487288 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:41.487698 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:41.487699 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:41.487868 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:56:41.487910 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:41.487911 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.488971 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.488972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:41.490369 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:41.490372 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.492423 (+ 2051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.492424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:41.492487 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:41.492892 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:56:41.492897 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:56:41.494232 (+ 1335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:56:41.495912 (+ 1680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:41.495914 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:41.495914 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:41.496188 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:41.496411 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:41.496412 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:41.496412 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:41.496476 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:41.496816 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:56:41.496817 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:56:41.496865 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 08:56:41.496915 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 08:56:41.497003 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185}
I20260809 08:56:41.497483 12790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:41.487088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39506 (local address 127.8.154.60:38091)
0809 08:56:41.487554 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:41.487556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:41.487562 (+ 6us) server_negotiation.cc:407] Connection header received
0809 08:56:41.487588 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:41.487589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:41.487597 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:41.487980 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:41.488036 (+ 56us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.488847 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.492110 (+ 3263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.492319 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.492661 (+ 342us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:41.494297 (+ 1636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:41.494298 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:41.494298 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:41.494305 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:41.495809 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:41.496277 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:41.496278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:41.496278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:41.496337 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:41.496555 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:41.496556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:41.496557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:41.496725 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:41.497222 (+ 497us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:41.497268 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 08:56:41.497352 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":111,"threads_started":1}
I20260809 08:56:41.496538 12468 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "af05a807602e41d3b424447c06734947"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52994
W20260809 08:56:41.498381 12810 catalog_manager.cc:1558] T 00000000000000000000000000000000 P af915e0b56c0482bbbad5799b7b6e87e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:56:41.498481 12810 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:56:41.499639 12519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af05a807602e41d3b424447c06734947: Bootstrap starting.
I20260809 08:56:41.498397 12670 master_runner.cc:186] Error getting master registration for 127.8.154.62:39313: OK, instance_id { permanent_uuid: "af05a807602e41d3b424447c06734947" instance_seqno: 1786265801142860 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 08:56:41.500124 12670 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 08:56:41.500406 12519 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af05a807602e41d3b424447c06734947: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:41.500742 12519 log.cc:824] T 00000000000000000000000000000000 P af05a807602e41d3b424447c06734947: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:41.501601 12519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af05a807602e41d3b424447c06734947: No bootstrap required, opened a new log
I20260809 08:56:41.502949 12519 raft_consensus.cc:348] T 00000000000000000000000000000000 P af05a807602e41d3b424447c06734947 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af05a807602e41d3b424447c06734947" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 39313 } } peers { permanent_uuid: "21af4c94d4ad4aafa50f8526e9663bea" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35915 } } peers { permanent_uuid: "af915e0b56c0482bbbad5799b7b6e87e" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 38091 } }
I20260809 08:56:41.503100 12519 raft_consensus.cc:729] T 00000000000000000000000000000000 P af05a807602e41d3b424447c06734947 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af05a807602e41d3b424447c06734947, State: Initialized, Role: FOLLOWER
I20260809 08:56:41.503189 12519 consensus_queue.cc:260] T 00000000000000000000000000000000 P af05a807602e41d3b424447c06734947 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af05a807602e41d3b424447c06734947" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 39313 } } peers { permanent_uuid: "21af4c94d4ad4aafa50f8526e9663bea" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35915 } } peers { permanent_uuid: "af915e0b56c0482bbbad5799b7b6e87e" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 38091 } }
I20260809 08:56:41.503525 12519 sys_catalog.cc:564] T 00000000000000000000000000000000 P af05a807602e41d3b424447c06734947 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:56:41.503521 12814 sys_catalog.cc:455] T 00000000000000000000000000000000 P af05a807602e41d3b424447c06734947 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af05a807602e41d3b424447c06734947" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 39313 } } peers { permanent_uuid: "21af4c94d4ad4aafa50f8526e9663bea" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35915 } } peers { permanent_uuid: "af915e0b56c0482bbbad5799b7b6e87e" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 38091 } } }
I20260809 08:56:41.503585 12468 raft_consensus.cc:482] T 00000000000000000000000000000000 P af05a807602e41d3b424447c06734947 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 08:56:41.503741 12468 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af05a807602e41d3b424447c06734947 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:41.504528 12468 raft_consensus.cc:504] T 00000000000000000000000000000000 P af05a807602e41d3b424447c06734947 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af05a807602e41d3b424447c06734947" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 39313 } } peers { permanent_uuid: "21af4c94d4ad4aafa50f8526e9663bea" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35915 } } peers { permanent_uuid: "af915e0b56c0482bbbad5799b7b6e87e" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 38091 } }
I20260809 08:56:41.504782 12468 leader_election.cc:288] T 00000000000000000000000000000000 P af05a807602e41d3b424447c06734947 [CANDIDATE]: Term 1 election: Requested vote from peers 21af4c94d4ad4aafa50f8526e9663bea (127.8.154.61:35915), af915e0b56c0482bbbad5799b7b6e87e (127.8.154.60:38091)
I20260809 08:56:41.506115 12748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "af05a807602e41d3b424447c06734947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "af915e0b56c0482bbbad5799b7b6e87e"
I20260809 08:56:41.506239 12748 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af915e0b56c0482bbbad5799b7b6e87e [term 0 FOLLOWER]: Advancing to term 1
W20260809 08:56:41.506590 12838 catalog_manager.cc:1558] T 00000000000000000000000000000000 P af05a807602e41d3b424447c06734947: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:56:41.506667 12838 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:56:41.507673 12748 raft_consensus.cc:2454] T 00000000000000000000000000000000 P af915e0b56c0482bbbad5799b7b6e87e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af05a807602e41d3b424447c06734947 in term 1.
I20260809 08:56:41.507994 12419 leader_election.cc:302] T 00000000000000000000000000000000 P af05a807602e41d3b424447c06734947 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af05a807602e41d3b424447c06734947, af915e0b56c0482bbbad5799b7b6e87e; no voters:
I20260809 08:56:41.508253 12814 sys_catalog.cc:458] T 00000000000000000000000000000000 P af05a807602e41d3b424447c06734947 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:56:41.506146 12390 master_runner.cc:186] Error getting master registration for 127.8.154.61:35915: OK, instance_id { permanent_uuid: "21af4c94d4ad4aafa50f8526e9663bea" instance_seqno: 1786265801310938 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 08:56:41.508369 12814 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af05a807602e41d3b424447c06734947 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:56:41.508395 12390 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 08:56:41.508420 12814 raft_consensus.cc:686] T 00000000000000000000000000000000 P af05a807602e41d3b424447c06734947 [term 1 LEADER]: Becoming Leader. State: Replica: af05a807602e41d3b424447c06734947, State: Running, Role: LEADER
I20260809 08:56:41.508497 12814 consensus_queue.cc:237] T 00000000000000000000000000000000 P af05a807602e41d3b424447c06734947 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af05a807602e41d3b424447c06734947" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 39313 } } peers { permanent_uuid: "21af4c94d4ad4aafa50f8526e9663bea" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35915 } } peers { permanent_uuid: "af915e0b56c0482bbbad5799b7b6e87e" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 38091 } }
I20260809 08:56:41.509033 12814 sys_catalog.cc:455] T 00000000000000000000000000000000 P af05a807602e41d3b424447c06734947 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af05a807602e41d3b424447c06734947. Latest consensus state: current_term: 1 leader_uuid: "af05a807602e41d3b424447c06734947" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af05a807602e41d3b424447c06734947" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 39313 } } peers { permanent_uuid: "21af4c94d4ad4aafa50f8526e9663bea" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35915 } } peers { permanent_uuid: "af915e0b56c0482bbbad5799b7b6e87e" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 38091 } } }
I20260809 08:56:41.509181 12814 sys_catalog.cc:458] T 00000000000000000000000000000000 P af05a807602e41d3b424447c06734947 [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:41.510056 12858 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:56:41.510254 12858 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:56:41.512130 12748 raft_consensus.cc:1261] T 00000000000000000000000000000000 P af915e0b56c0482bbbad5799b7b6e87e [term 1 FOLLOWER]: Refusing update from remote peer af05a807602e41d3b424447c06734947: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:56:41.512529 12853 consensus_queue.cc:1035] T 00000000000000000000000000000000 P af05a807602e41d3b424447c06734947 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af915e0b56c0482bbbad5799b7b6e87e" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 38091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:56:41.513864 12794 sys_catalog.cc:455] T 00000000000000000000000000000000 P af915e0b56c0482bbbad5799b7b6e87e [sys.catalog]: SysCatalogTable state changed. Reason: New leader af05a807602e41d3b424447c06734947. Latest consensus state: current_term: 1 leader_uuid: "af05a807602e41d3b424447c06734947" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af05a807602e41d3b424447c06734947" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 39313 } } peers { permanent_uuid: "21af4c94d4ad4aafa50f8526e9663bea" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35915 } } peers { permanent_uuid: "af915e0b56c0482bbbad5799b7b6e87e" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 38091 } } }
I20260809 08:56:41.513952 12794 sys_catalog.cc:458] T 00000000000000000000000000000000 P af915e0b56c0482bbbad5799b7b6e87e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:56:41.526005 12525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:41.507056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52996 (local address 127.8.154.62:39313)
0809 08:56:41.507097 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:41.507099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:41.516221 (+ 9122us) server_negotiation.cc:407] Connection header received
0809 08:56:41.516286 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:41.516287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:41.516300 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:41.516359 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:41.517556 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.518311 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.519868 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.520178 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.520242 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:41.522501 (+ 2259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:41.522502 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:41.522503 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:41.522511 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:41.524266 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:41.524891 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:41.524892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:41.524893 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:41.524952 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:41.525257 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:41.525258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:41.525258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:41.525405 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:41.525476 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:41.525822 (+ 346us) server_negotiation.cc:299] Negotiation successful
0809 08:56:41.525904 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 08:56:41.527796 12794 sys_catalog.cc:455] T 00000000000000000000000000000000 P af915e0b56c0482bbbad5799b7b6e87e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af05a807602e41d3b424447c06734947" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af05a807602e41d3b424447c06734947" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 39313 } } peers { permanent_uuid: "21af4c94d4ad4aafa50f8526e9663bea" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35915 } } peers { permanent_uuid: "af915e0b56c0482bbbad5799b7b6e87e" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 38091 } } }
I20260809 08:56:41.527917 12794 sys_catalog.cc:458] T 00000000000000000000000000000000 P af915e0b56c0482bbbad5799b7b6e87e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:56:41.528640 12814 sys_catalog.cc:455] T 00000000000000000000000000000000 P af05a807602e41d3b424447c06734947 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af05a807602e41d3b424447c06734947" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af05a807602e41d3b424447c06734947" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 39313 } } peers { permanent_uuid: "21af4c94d4ad4aafa50f8526e9663bea" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35915 } } peers { permanent_uuid: "af915e0b56c0482bbbad5799b7b6e87e" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 38091 } } }
I20260809 08:56:41.528817 12814 sys_catalog.cc:458] T 00000000000000000000000000000000 P af05a807602e41d3b424447c06734947 [sys.catalog]: This master's current role is: LEADER
I20260809 08:56:41.529086 12864 mvcc.cc:205] Tried to move back new op lower bound from 7316544722990256128 to 7316544722979930112. Current Snapshot: MvccSnapshot[applied={T|T < 7316544722990256128 or (T in {7316544722990256128})}]
I20260809 08:56:41.533072 12790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:41.509965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39512 (local address 127.8.154.60:38091)
0809 08:56:41.510027 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:41.510029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:41.523881 (+ 13852us) server_negotiation.cc:407] Connection header received
0809 08:56:41.523990 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:41.523991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:41.524004 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:41.524037 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:41.524805 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.525550 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.526615 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.526914 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.526976 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:41.528557 (+ 1581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:41.528558 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:41.528559 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:41.528566 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:41.531045 (+ 2479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:41.532096 (+ 1051us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:41.532097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:41.532097 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:41.532154 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:41.532345 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:41.532346 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:41.532346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:41.532476 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:41.532592 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:41.532867 (+ 275us) server_negotiation.cc:299] Negotiation successful
0809 08:56:41.532958 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 08:56:41.533499 12664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:41.509555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33952 (local address 127.8.154.61:35915)
0809 08:56:41.509600 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:41.509602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:41.516380 (+ 6778us) server_negotiation.cc:407] Connection header received
0809 08:56:41.520156 (+ 3776us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:41.520158 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:41.520172 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:41.520206 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:41.521032 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.521772 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.523013 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.523567 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.523647 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:41.529915 (+ 6268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:41.529917 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:41.529917 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:41.529925 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:41.531565 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:41.532626 (+ 1061us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:41.532627 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:41.532628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:41.532686 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:41.532893 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:41.532894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:41.532895 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:41.533042 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:41.533117 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:41.533318 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 08:56:41.533405 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 08:56:41.534363 12858 catalog_manager.cc:1347] Generated new cluster ID: db867ca4c9f6474e8768b545acb8f4e0
I20260809 08:56:41.534492 12858 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:56:41.544986 12858 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:56:41.545732 12858 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:56:41.546823 12660 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.8.154.60" port: 38091 } attempt: 3
Aug 09 08:56:41 dist-test-slave-t3q3 krb5kdc[12373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265801, etypes {rep=17 tkt=17 ses=17}, kudu/127.8.154.61@KRBTEST.COM for kudu/127.8.154.60@KRBTEST.COM
I20260809 08:56:41.554919 12858 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af05a807602e41d3b424447c06734947: Generated new TSK 0
I20260809 08:56:41.555138 12858 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:56:41.556607 12663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:41.547112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.8.154.60:38091 (local address 127.0.0.1:39526)
0809 08:56:41.547162 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 08:56:41.547174 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:56:41.547216 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:56:41.547369 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:56:41.547370 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:56:41.547573 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:56:41.547629 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:41.547632 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.548585 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.548587 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:41.549354 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:56:41.549357 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.549907 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.549908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:56:41.549988 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:41.550511 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:56:41.550517 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:56:41.552052 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:56:41.555033 (+ 2981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:41.555035 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:41.555036 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:41.555253 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:41.555721 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:56:41.555722 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:56:41.555723 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:56:41.555800 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:56:41.556298 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:56:41.556299 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:56:41.556345 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 08:56:41.556393 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 08:56:41.556497 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 08:56:41.561410 12790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:41.547155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39526 (local address 127.8.154.60:38091)
0809 08:56:41.547215 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:41.547217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:41.547227 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:56:41.547284 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:41.547285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:41.547295 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:41.547705 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:41.547752 (+ 47us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.548507 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.549475 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:41.549801 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:41.551119 (+ 1318us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:41.552167 (+ 1048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:56:41.552168 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:56:41.552168 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:56:41.552175 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:56:41.554897 (+ 2722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:41.555488 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:41.555489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:41.555490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:41.555615 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:56:41.555888 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:56:41.555889 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:56:41.555889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:56:41.556178 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:56:41.557067 (+ 889us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:41.557156 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 08:56:41.557327 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 08:56:41.563263 12660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21af4c94d4ad4aafa50f8526e9663bea: Bootstrap starting.
I20260809 08:56:41.564014 12660 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21af4c94d4ad4aafa50f8526e9663bea: Neither blocks nor log segments found. Creating new log.
I20260809 08:56:41.564358 12660 log.cc:824] T 00000000000000000000000000000000 P 21af4c94d4ad4aafa50f8526e9663bea: Log is configured to *not* fsync() on all Append() calls
I20260809 08:56:41.565835 12660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21af4c94d4ad4aafa50f8526e9663bea: No bootstrap required, opened a new log
I20260809 08:56:41.567863 12660 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21af4c94d4ad4aafa50f8526e9663bea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af05a807602e41d3b424447c06734947" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 39313 } } peers { permanent_uuid: "21af4c94d4ad4aafa50f8526e9663bea" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35915 } } peers { permanent_uuid: "af915e0b56c0482bbbad5799b7b6e87e" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 38091 } }
I20260809 08:56:41.568002 12660 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21af4c94d4ad4aafa50f8526e9663bea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21af4c94d4ad4aafa50f8526e9663bea, State: Initialized, Role: FOLLOWER
I20260809 08:56:41.568099 12660 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21af4c94d4ad4aafa50f8526e9663bea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af05a807602e41d3b424447c06734947" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 39313 } } peers { permanent_uuid: "21af4c94d4ad4aafa50f8526e9663bea" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35915 } } peers { permanent_uuid: "af915e0b56c0482bbbad5799b7b6e87e" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 38091 } }
I20260809 08:56:41.568411 12660 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21af4c94d4ad4aafa50f8526e9663bea [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:56:41.569885 12610 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21af4c94d4ad4aafa50f8526e9663bea [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:56:41.569715 12611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "af05a807602e41d3b424447c06734947" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "21af4c94d4ad4aafa50f8526e9663bea"
I20260809 08:56:41.570794 12610 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 21af4c94d4ad4aafa50f8526e9663bea [term 1 FOLLOWER]: Refusing update from remote peer af05a807602e41d3b424447c06734947: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:56:41.571226 12867 consensus_queue.cc:1035] T 00000000000000000000000000000000 P af05a807602e41d3b424447c06734947 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21af4c94d4ad4aafa50f8526e9663bea" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:56:41.571764 12611 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 21af4c94d4ad4aafa50f8526e9663bea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af05a807602e41d3b424447c06734947 in term 1.
I20260809 08:56:41.573863 12901 mvcc.cc:205] Tried to move back new op lower bound from 7316544723163377664 to 7316544722979930112. Current Snapshot: MvccSnapshot[applied={T|T < 7316544722990256128}]
I20260809 08:56:41.575006 12905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21af4c94d4ad4aafa50f8526e9663bea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "af05a807602e41d3b424447c06734947" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af05a807602e41d3b424447c06734947" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 39313 } } peers { permanent_uuid: "21af4c94d4ad4aafa50f8526e9663bea" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35915 } } peers { permanent_uuid: "af915e0b56c0482bbbad5799b7b6e87e" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 38091 } } }
I20260809 08:56:41.575089 12905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21af4c94d4ad4aafa50f8526e9663bea [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:56:41.575133 12905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21af4c94d4ad4aafa50f8526e9663bea [sys.catalog]: SysCatalogTable state changed. Reason: New leader af05a807602e41d3b424447c06734947. Latest consensus state: current_term: 1 leader_uuid: "af05a807602e41d3b424447c06734947" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af05a807602e41d3b424447c06734947" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 39313 } } peers { permanent_uuid: "21af4c94d4ad4aafa50f8526e9663bea" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35915 } } peers { permanent_uuid: "af915e0b56c0482bbbad5799b7b6e87e" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 38091 } } }
I20260809 08:56:41.575183 12905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21af4c94d4ad4aafa50f8526e9663bea [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:56:41.575217 12905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21af4c94d4ad4aafa50f8526e9663bea [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af05a807602e41d3b424447c06734947" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af05a807602e41d3b424447c06734947" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 39313 } } peers { permanent_uuid: "21af4c94d4ad4aafa50f8526e9663bea" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35915 } } peers { permanent_uuid: "af915e0b56c0482bbbad5799b7b6e87e" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 38091 } } }
I20260809 08:56:41.575261 12905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21af4c94d4ad4aafa50f8526e9663bea [sys.catalog]: This master's current role is: FOLLOWER
W20260809 08:56:41.575976 12893 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 21af4c94d4ad4aafa50f8526e9663bea: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:56:41.576174 12893 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260809 08:56:41.577061 12525 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:56:41.574120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53002 (local address 127.8.154.62:39313)
0809 08:56:41.574159 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:41.574161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:41.576159 (+ 1998us) server_negotiation.cc:407] Connection header received
0809 08:56:41.576541 (+ 382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:41.576542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:41.576554 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:41.576584 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:41.576999 (+ 415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53002: BlockingRecv error: recv got EOF from 127.0.0.1:53002 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 08:56:41.577742 12664 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:56:41.576056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33956 (local address 127.8.154.61:35915)
0809 08:56:41.576112 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:41.576114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:41.577247 (+ 1133us) server_negotiation.cc:407] Connection header received
0809 08:56:41.577315 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:41.577316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:41.577326 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:41.577385 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:41.577693 (+ 308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33956: BlockingRecv error: recv got EOF from 127.0.0.1:33956 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 08:56:41.578508 12790 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:56:41.576417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39536 (local address 127.8.154.60:38091)
0809 08:56:41.576450 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:41.576452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:41.578030 (+ 1578us) server_negotiation.cc:407] Connection header received
0809 08:56:41.578125 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:41.578126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:41.578136 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:41.578165 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:41.578450 (+ 285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39536: BlockingRecv error: recv got EOF from 127.0.0.1:39536 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 08:56:41.580847 12664 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:56:41.579728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33972 (local address 127.8.154.61:35915)
0809 08:56:41.579762 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:41.579764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:41.580327 (+ 563us) server_negotiation.cc:407] Connection header received
0809 08:56:41.580433 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:41.580434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:41.580450 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:41.580479 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:41.580805 (+ 326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33972: BlockingRecv error: recv got EOF from 127.0.0.1:33972 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 08:56:41.583745 12664 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:56:41.583024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33988 (local address 127.8.154.61:35915)
0809 08:56:41.583054 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:41.583056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:41.583272 (+ 216us) server_negotiation.cc:407] Connection header received
0809 08:56:41.583363 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:41.583364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:41.583373 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:41.583401 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:41.583707 (+ 306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33988: BlockingRecv error: recv got EOF from 127.0.0.1:33988 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 08:56:41.587424 12664 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:56:41.586743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33996 (local address 127.8.154.61:35915)
0809 08:56:41.586776 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:41.586777 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:41.586975 (+ 198us) server_negotiation.cc:407] Connection header received
0809 08:56:41.587065 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:41.587066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:41.587076 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:41.587103 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:41.587386 (+ 283us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33996: BlockingRecv error: recv got EOF from 127.0.0.1:33996 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 08:56:41.593235 12664 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:56:41.592501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34000 (local address 127.8.154.61:35915)
0809 08:56:41.592545 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:41.592547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:41.592755 (+ 208us) server_negotiation.cc:407] Connection header received
0809 08:56:41.592845 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:41.592846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:41.592854 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:41.592882 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:41.593196 (+ 314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34000: BlockingRecv error: recv got EOF from 127.0.0.1:34000 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260809 08:56:41.603087 12664 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:56:41.602312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34006 (local address 127.8.154.61:35915)
0809 08:56:41.602346 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:41.602348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:41.602584 (+ 236us) server_negotiation.cc:407] Connection header received
0809 08:56:41.602683 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:41.602684 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:41.602694 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:41.602724 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:41.603047 (+ 323us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34006: BlockingRecv error: recv got EOF from 127.0.0.1:34006 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 08:56:41.621201 12664 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:56:41.620341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34022 (local address 127.8.154.61:35915)
0809 08:56:41.620379 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:41.620381 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:41.620654 (+ 273us) server_negotiation.cc:407] Connection header received
0809 08:56:41.620755 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:41.620757 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:41.620767 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:41.620797 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:41.621150 (+ 353us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34022: BlockingRecv error: recv got EOF from 127.0.0.1:34022 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 08:56:41.655616 12664 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:56:41.654592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34028 (local address 127.8.154.61:35915)
0809 08:56:41.654638 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:41.654640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:41.654933 (+ 293us) server_negotiation.cc:407] Connection header received
0809 08:56:41.655074 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:41.655075 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:41.655088 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:41.655120 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:41.655554 (+ 434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34028: BlockingRecv error: recv got EOF from 127.0.0.1:34028 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 08:56:41.722183 12664 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:56:41.721360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34036 (local address 127.8.154.61:35915)
0809 08:56:41.721414 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:41.721417 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:41.721603 (+ 186us) server_negotiation.cc:407] Connection header received
0809 08:56:41.721669 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:41.721671 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:41.721685 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:41.721768 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:41.722117 (+ 349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34036: BlockingRecv error: recv got EOF from 127.0.0.1:34036 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260809 08:56:41.852272 12664 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:56:41.851502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34048 (local address 127.8.154.61:35915)
0809 08:56:41.851569 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:41.851572 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:41.851713 (+ 141us) server_negotiation.cc:407] Connection header received
0809 08:56:41.851756 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:41.851757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:41.851768 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:41.851836 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:41.852187 (+ 351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34048: BlockingRecv error: recv got EOF from 127.0.0.1:34048 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 08:56:41.973606 12853 sys_catalog.cc:455] T 00000000000000000000000000000000 P af05a807602e41d3b424447c06734947 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af05a807602e41d3b424447c06734947" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af05a807602e41d3b424447c06734947" member_type: VOTER last_known_addr { host: "127.8.154.62" port: 39313 } } peers { permanent_uuid: "21af4c94d4ad4aafa50f8526e9663bea" member_type: VOTER last_known_addr { host: "127.8.154.61" port: 35915 } } peers { permanent_uuid: "af915e0b56c0482bbbad5799b7b6e87e" member_type: VOTER last_known_addr { host: "127.8.154.60" port: 38091 } } }
I20260809 08:56:41.973744 12853 sys_catalog.cc:458] T 00000000000000000000000000000000 P af05a807602e41d3b424447c06734947 [sys.catalog]: This master's current role is: LEADER
W20260809 08:56:42.110440 12664 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:56:42.109536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34058 (local address 127.8.154.61:35915)
0809 08:56:42.109632 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:42.109633 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:42.109671 (+ 38us) server_negotiation.cc:407] Connection header received
0809 08:56:42.109699 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:42.109700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:42.109711 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:42.109757 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:56:42.110377 (+ 620us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34058: BlockingRecv error: recv got EOF from 127.0.0.1:34058 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 08:56:42.498977 12810 catalog_manager.cc:1259] Loaded cluster ID: db867ca4c9f6474e8768b545acb8f4e0
I20260809 08:56:42.499063 12810 catalog_manager.cc:1552] T 00000000000000000000000000000000 P af915e0b56c0482bbbad5799b7b6e87e: loading cluster ID for follower catalog manager: success
I20260809 08:56:42.499859 12810 catalog_manager.cc:1574] T 00000000000000000000000000000000 P af915e0b56c0482bbbad5799b7b6e87e: acquiring CA information for follower catalog manager: success
I20260809 08:56:42.500142 12810 catalog_manager.cc:1602] T 00000000000000000000000000000000 P af915e0b56c0482bbbad5799b7b6e87e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 08:56:42.576591 12893 catalog_manager.cc:1259] Loaded cluster ID: db867ca4c9f6474e8768b545acb8f4e0
I20260809 08:56:42.576686 12893 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 21af4c94d4ad4aafa50f8526e9663bea: loading cluster ID for follower catalog manager: success
I20260809 08:56:42.577565 12893 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 21af4c94d4ad4aafa50f8526e9663bea: acquiring CA information for follower catalog manager: success
I20260809 08:56:42.577921 12893 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 21af4c94d4ad4aafa50f8526e9663bea: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 08:56:42.626194 13011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:42.623624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34072 (local address 127.8.154.61:35915)
0809 08:56:42.623737 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:42.623739 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:42.623751 (+ 12us) server_negotiation.cc:407] Connection header received
0809 08:56:42.623837 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:42.623838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:42.623889 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:42.623970 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:56:42.624170 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:42.624696 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:42.625448 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:42.625653 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:42.625707 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:42.625846 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:56:42.625960 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:56:42.626020 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:42.626054 (+ 34us) server_negotiation.cc:299] Negotiation successful
0809 08:56:42.626071 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":64,"threads_started":1}
I20260809 08:56:42.630414 13023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:42.628027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39552 (local address 127.8.154.60:38091)
0809 08:56:42.628171 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:42.628173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:42.628182 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:56:42.628223 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:42.628224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:42.628260 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:42.628329 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:56:42.628429 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:42.628912 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:42.629606 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:42.629800 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:42.629852 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:42.630083 (+ 231us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:56:42.630181 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:56:42.630241 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:42.630285 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 08:56:42.630301 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":80,"threads_started":1}
I20260809 08:56:42.635277 13033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:42.632288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53012 (local address 127.8.154.62:39313)
0809 08:56:42.632443 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:42.632445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:42.632454 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:56:42.632505 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:42.632506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:42.632559 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:42.632659 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:56:42.632805 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:42.633400 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:42.634315 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:42.634630 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:42.634691 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:42.634930 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:56:42.635051 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:56:42.635110 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:42.635170 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 08:56:42.635194 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":59,"threads_started":1}
I20260809 08:56:42.635623 13023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:42.632567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39554 (local address 127.8.154.60:38091)
0809 08:56:42.632627 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:42.632629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:42.632782 (+ 153us) server_negotiation.cc:407] Connection header received
0809 08:56:42.632838 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:42.632839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:42.632849 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:42.632891 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:56:42.633051 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:42.633758 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:42.634683 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:42.634951 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:42.635017 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:42.635190 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:56:42.635221 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:56:42.635250 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:42.635540 (+ 290us) server_negotiation.cc:299] Negotiation successful
0809 08:56:42.635562 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 08:56:42.635627 13011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:56:42.632281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34084 (local address 127.8.154.61:35915)
0809 08:56:42.632389 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0809 08:56:42.632391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:56:42.632648 (+ 257us) server_negotiation.cc:407] Connection header received
0809 08:56:42.632750 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:56:42.632752 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:56:42.632765 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:56:42.632816 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:56:42.632978 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:42.633746 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:42.634816 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:56:42.635101 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:56:42.635160 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:56:42.635351 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:56:42.635389 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:56:42.635444 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 08:56:42.635515 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 08:56:42.635537 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 08:56:42.637229 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 12390
I20260809 08:56:42.650399 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 12530
I20260809 08:56:42.659333 8808 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskBbavBz/build/release/bin/kudu with pid 12670
2026-08-09T08:56:42Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3022 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6130 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (282713 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST